Re: Evolutionary User Strategy - A Compromise
On Thursday 13 July 2006 12:04, Anthony Youngman wrote:
Might be an idea to look at Antlr then ...
I don't know how effective it would be, but part of the purpose behind
the v3 rewrite is to increase the number of languages that Antlr can
generate. If you can define the grammar in Antlr it could then chuck out
a lexer/parser written in Scheme :-) Is the current lily parser
hand-written?
no, it's written in bison. The lexer invokes an external parser to parse
Scheme expression when needed.
And yes, preservation of white space is apparently fairly easy.
Although, I'm half inclined to say that if you convert-ly a piece,
then it will or can output stuff according to a pretty print standard.
question is whether it's possible to write a grammar that is useful both for
convert-ly and paper production at once, or if it's a better idea to use a
simplified grammar for convert-ly. E.g.: in { \foo c4 c2 } it's difficult to
know whether c4 and/or c2 are parameters to the function foo, or if foo just
is a variable -- it might be that foo has been defined in an \included file.
For convert-ly purposes, it might be sensible to always treat \foo as a
variable, because that makes life simpler.
Take a look at Antlr - www.antlr.org. I'd go straight to v3, which is
due for release this summer. What you would need to do is define the
lily grammar using a BNF style notation (I think the technical term for
the style of grammar is LA or LLA). Antlr itself is written in Java -
run it over your grammar and it will spit out a lexer and parser for
you. You could either use the current C++ templates which would create
your lexer and parser in C++ for feeding the results to Scheme, or write
a bunch of Scheme templates and output the lexer and parser directly in
Scheme. I don't mean to teach grandma to suck eggs but it sounds (from
the implication that the parser is mixed-language) that you're not using
a compiler-compiler such as flex/Bison, PCCTS or Antlr.
And of course, going down this route, things head towards the typical
nix toolkit approach :-) Grammars to input lily ASTs in various
versions, grammars to output pretty printed or converted .ly files in
various versions, grammars to transform ASTs especially ones generated
from eg abc, Sibelius, Personal Composer files :-)
I think it's easier to import from other formats directly to .ly, or to a
known format such as MusicXML.
--
Erik
___
lilypond-user mailing list
lilypond-user@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-user
Re: Evolutionary User Strategy - A Compromise
The current parser is implemented using flex/bison. /Mats Anthony Youngman wrote: Might be an idea to look at Antlr then ... I don't know how effective it would be, but part of the purpose behind the v3 rewrite is to increase the number of languages that Antlr can generate. If you can define the grammar in Antlr it could then chuck out a lexer/parser written in Scheme :-) Is the current lily parser hand-written? At the expense of some upfront work, this could save a lot of effort downstream if so. It does sound very much as though what you want to do, is what Antlr is designed to do ... you say Scheme rocks for tree manipulation? Well Antlr was DESIGNED for tree manipulation (and other related tasks :-) ... And yes, preservation of white space is apparently fairly easy. Although, I'm half inclined to say that if you convert-ly a piece, then it will or can output stuff according to a pretty print standard. Take a look at Antlr - www.antlr.org. I'd go straight to v3, which is due for release this summer. What you would need to do is define the lily grammar using a BNF style notation (I think the technical term for the style of grammar is LA or LLA). Antlr itself is written in Java - run it over your grammar and it will spit out a lexer and parser for you. You could either use the current C++ templates which would create your lexer and parser in C++ for feeding the results to Scheme, or write a bunch of Scheme templates and output the lexer and parser directly in Scheme. I don't mean to teach grandma to suck eggs but it sounds (from the implication that the parser is mixed-language) that you're not using a compiler-compiler such as flex/Bison, PCCTS or Antlr. And of course, going down this route, things head towards the typical nix toolkit approach :-) Grammars to input lily ASTs in various versions, grammars to output pretty printed or converted .ly files in various versions, grammars to transform ASTs especially ones generated from eg abc, Sibelius, Personal Composer files :-) Just link the appropriate sequence of tools and away you go ... I can see it being a lot of up-front work. The question is, will it save more than that in future ... and I think your answer to is it possible is yes, just is it worth it? Cheers, Wol -Original Message- From: Erik Sandberg [mailto:[EMAIL PROTECTED] Sent: 12 July 2006 19:08 To: lilypond-user@gnu.org Cc: Anthony Youngman Subject: Re: Evolutionary User Strategy - A Compromise On Wednesday 12 July 2006 17:22, Anthony Youngman wrote: I don't really understand grammars etc (which is why my DATABASIC thing is on/off :-). But from my experience with Antlr, I don't see why you should lose stuff. Your PEG article mentions ASTs. I don't see that converting a .ly file into an AST can be that hard. So, for example, we write a Antlr grammar that creates a lexer/parser that turns the .ly into an AST. We now write another grammar that converts the AST to a .ly file. A problem here is code duplication; it takes some effort to maintain two parsers instead of one. I think it will be difficult to automatically test that the current antlr parser corresponds well with the actual grammar the current lilypond version uses. I have been thinking about moving lily's entire parser out to Scheme; this way we could keep one old parser for each version, and use it to generate an AST, which then is converted nicely using rules written in Scheme (Scheme rocks when it comes to tree manipulation). I'm not sure if it's possible though. BTW: Will your solution handle whitespace nicely? -- = Mats Bengtsson Signal Processing Signals, Sensors and Systems Royal Institute of Technology SE-100 44 STOCKHOLM Sweden Phone: (+46) 8 790 8463 Fax: (+46) 8 790 7260 Email: [EMAIL PROTECTED] WWW: http://www.s3.kth.se/~mabe = ___ lilypond-user mailing list lilypond-user@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-user
Re: Evolutionary User Strategy - A Compromise
Maybe a distinction should be made between bug fixes and other changes, e.g new features and syntax changes. I would not expect the developers to preserve bugs, but beyond that I think it is reasonable to hope that a file I create in version 3.0 should look the same if I process it using version 3.1. Or at least that the instructions I used should still work and should have the same meaning. When I use a specific tweak, such as adding an offset to a piece of markup, I don't expect it to be particularly stable - if the system breaks change, for example, then I have to review such tweaks to check that they still have the right effect. So I can live with having to do that for new versions. All I'm trying to do here is suggest an approach that makes it easier for users to continue to use Lilypond when new versions come out. The current lack of stability is a major barrier to using Lilypond for some people. Colin Erik Sandberg-2 wrote: On Wednesday 12 July 2006 12:00, Colin Wilding wrote: This is an important dilemma for many users, I think - we want to have all the fixes and features in each new version, but find it frustrating when music produced in earlier versions needs time-consuming manual editing to upgrade. Can I suggest a compromise? I accept that Lilypond has been evolving rapidly and feel privileged to have been able to use it (and contribute comments) during that evolution. At some point, though, the evolution should slow down: the developers should feel happy with the basic structure and syntax and the users should be able to expect that music written for today?s version will look the same in tomorrow?s. What do you mean by 'the same'? Is it OK if the output suddenly looks better because a bug has been fixed? If yes, then you still have the problem that an upgrade may break your tweaks that work around layout bugs. If no, then development will be very difficult because all bugs have to be preserved. If there's sufficient community interest in maintaining and developing a conservative version of lilypond, then anyone can make a fork; however, I doubt any of the current developers would join. Lily is currently at a pretty early stage of development IMHO. How about making a resolution that from version 3.0 onwards Lilypond will be backwards-compatible: in other words, the current version will correctly display a file written in any earlier version 3.x without the need for conversion. I think 2.4 files are fairly forward compatible; there have been no major changes that I know of. There's also the question of what you mean by compatibility: Very advanced tweaks usually rely on the way lily's internals are organised, which may change over time. Since lily contains a Turing-complete programming language, for some language updates it is thereby _impossible_ to create a script that upgrades _all_ .ly files perfectly. There are also deprecated features that are dropped sometimes, this causes a kind of backward incompatibility. -- Erik ___ lilypond-user mailing list lilypond-user@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-user -- View this message in context: http://www.nabble.com/Evolutionary-User-Strategery-tf1906879.html#a5304610 Sent from the Gnu - Lilypond - User forum at Nabble.com. ___ lilypond-user mailing list lilypond-user@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-user
RE: Evolutionary User Strategy - A Compromise
Might be an idea to look at Antlr then ... I don't know how effective it would be, but part of the purpose behind the v3 rewrite is to increase the number of languages that Antlr can generate. If you can define the grammar in Antlr it could then chuck out a lexer/parser written in Scheme :-) Is the current lily parser hand-written? At the expense of some upfront work, this could save a lot of effort downstream if so. It does sound very much as though what you want to do, is what Antlr is designed to do ... you say Scheme rocks for tree manipulation? Well Antlr was DESIGNED for tree manipulation (and other related tasks :-) ... And yes, preservation of white space is apparently fairly easy. Although, I'm half inclined to say that if you convert-ly a piece, then it will or can output stuff according to a pretty print standard. Take a look at Antlr - www.antlr.org. I'd go straight to v3, which is due for release this summer. What you would need to do is define the lily grammar using a BNF style notation (I think the technical term for the style of grammar is LA or LLA). Antlr itself is written in Java - run it over your grammar and it will spit out a lexer and parser for you. You could either use the current C++ templates which would create your lexer and parser in C++ for feeding the results to Scheme, or write a bunch of Scheme templates and output the lexer and parser directly in Scheme. I don't mean to teach grandma to suck eggs but it sounds (from the implication that the parser is mixed-language) that you're not using a compiler-compiler such as flex/Bison, PCCTS or Antlr. And of course, going down this route, things head towards the typical nix toolkit approach :-) Grammars to input lily ASTs in various versions, grammars to output pretty printed or converted .ly files in various versions, grammars to transform ASTs especially ones generated from eg abc, Sibelius, Personal Composer files :-) Just link the appropriate sequence of tools and away you go ... I can see it being a lot of up-front work. The question is, will it save more than that in future ... and I think your answer to is it possible is yes, just is it worth it? Cheers, Wol -Original Message- From: Erik Sandberg [mailto:[EMAIL PROTECTED] Sent: 12 July 2006 19:08 To: lilypond-user@gnu.org Cc: Anthony Youngman Subject: Re: Evolutionary User Strategy - A Compromise On Wednesday 12 July 2006 17:22, Anthony Youngman wrote: I don't really understand grammars etc (which is why my DATABASIC thing is on/off :-). But from my experience with Antlr, I don't see why you should lose stuff. Your PEG article mentions ASTs. I don't see that converting a .ly file into an AST can be that hard. So, for example, we write a Antlr grammar that creates a lexer/parser that turns the .ly into an AST. We now write another grammar that converts the AST to a .ly file. A problem here is code duplication; it takes some effort to maintain two parsers instead of one. I think it will be difficult to automatically test that the current antlr parser corresponds well with the actual grammar the current lilypond version uses. I have been thinking about moving lily's entire parser out to Scheme; this way we could keep one old parser for each version, and use it to generate an AST, which then is converted nicely using rules written in Scheme (Scheme rocks when it comes to tree manipulation). I'm not sure if it's possible though. BTW: Will your solution handle whitespace nicely? -- Erik * * This transmission is intended for the named recipient only. It may contain private and confidential information. If this has come to you in error you must not act on anything disclosed in it, nor must you copy it, modify it, disseminate it in any way, or show it to anyone. Please e-mail the sender to inform us of the transmission error or telephone ECA International immediately and delete the e-mail from your information system. Telephone numbers for ECA International offices are: Sydney +61 (0)2 8272 5300, Hong Kong + 852 2121 2388, London +44 (0)20 7351 5000 and New York +1 212 582 2333. * * ___ lilypond-user mailing list lilypond-user@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-user
Re: Evolutionary User Strategy - A Compromise
This is an important dilemma for many users, I think - we want to have all the fixes and features in each new version, but find it frustrating when music produced in earlier versions needs time-consuming manual editing to upgrade. Can I suggest a compromise? I accept that Lilypond has been evolving rapidly and feel privileged to have been able to use it (and contribute comments) during that evolution. At some point, though, the evolution should slow down: the developers should feel happy with the basic structure and syntax and the users should be able to expect that music written for today?s version will look the same in tomorrow?s. How about making a resolution that from version 3.0 onwards Lilypond will be backwards-compatible: in other words, the current version will correctly display a file written in any earlier version 3.x without the need for conversion. For the developers that would mean retaining the behaviour of the earlier versions, but only back as far as 3.0. For the users it would mean that once they had updated their files to 3.0 they could continue to use them without further modification if they wished. If the backward compatibilty subsequently became a burden then the developers could start again with version 4.0 and write a convert-ly just from 3.x to 4.0. Colin Fairchild wrote: LilyPonders - Users dealing with LilyPond as it evolves are presented with difficult choices, none good. If one knows a score will be completed quickly, never be revisited, and components of the code structure never will be reused, the choice is easy: save and share graphics only, dump the ly files for completed scores, and move on to the latest stable version. I'm not that prescient. Constantly adopting the latest version allows use of the latest features, feedback helps the developers, and the developers provide support. However, modifying 'finished' scores to be acceptable by the latest version is not reasonable. Upgrade modifications require significant effort. The convert-ly program helps, but misses a lot. It is tempting to select a stable version and stick with it. Scores can be revisited easily. Syntax and semantics are stable. Downside: the feature set never gets better and support will fade away unless a sufficient number of users make the same choice and help each other. It is possible to maintain multiple LilyPond versions. This allows revisiting old scores, but at a price. The operational environment becomes more and more confusing as versions accumulate. I have coded tens of scores, most with version 2.4.6, and spent several hours attempting to move just one (a smaller one) of them to 2.8.5. After using convert-ly and correcting for its errors, much remains to be done, especially to position the right text font. Conclusion: repetitive conversion of scores is untenable. The only reasonable solution is to maintain upward compatibility in the LilyPond processor. New features should be added without changing existing syntax. If it is deemed absolutely necessary to change semantics or define conflicting syntax, provide for optional interpretations based on the version specified. Older ly files should generate consistent results as LilyPond migrates. More exhaustive regression tests are necessary. If you can identify a better way, or have other comments, please respond. - Bruce ___ lilypond-user mailing list lilypond-user@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-user -- View this message in context: http://www.nabble.com/Evolutionary-User-Strategery-tf1906879.html#a5285941 Sent from the Gnu - Lilypond - User forum at Nabble.com. ___ lilypond-user mailing list lilypond-user@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-user
RE: Evolutionary User Strategy - A Compromise
Something I thought of (having seen the comment about convert-ly using grep ...) I've got an on-off thing about writing a DATABASIC compiler (never mind) and have come across a tool called Antlr. It is a compiler-compiler and generates lexers, parsers and treeparsers. IF someone wants to put the effort in, it may (or may not be) easy to define various grammars to read in and chuck out different music formats. It sounds as though a lot of the problems (like the swap between and for example) would be easy. Especially given lilypond's structure it looks like it would be fairly easy to define grammars which can read in or chuck out different lily version syntaxes, even the \addLyrics / \oldAddLyrics thing maybe. The disadvantage of going down this route is I can see maintenance of convert-ly becoming a big task. The advantage is that adding a new input or output syntax is simply a matter of adding another grammar definition. Only rarely are we likely to have to write a grammar to transform convert-ly's internal representation - just when either (a) we add a completely new 3rd-party format to the tool, or if there's a major syntax change in lily that requires a paradigm shift ... Cheers, Wol -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] .org] On Behalf Of Colin Wilding Sent: 12 July 2006 11:01 To: lilypond-user@gnu.org Subject: Re: Evolutionary User Strategy - A Compromise This is an important dilemma for many users, I think - we want to have all the fixes and features in each new version, but find it frustrating when music produced in earlier versions needs time-consuming manual editing to upgrade. Can I suggest a compromise? I accept that Lilypond has been evolving rapidly and feel privileged to have been able to use it (and contribute comments) during that evolution. At some point, though, the evolution should slow down: the developers should feel happy with the basic structure and syntax and the users should be able to expect that music written for today?s version will look the same in tomorrow?s. How about making a resolution that from version 3.0 onwards Lilypond will be backwards-compatible: in other words, the current version will correctly display a file written in any earlier version 3.x without the need for conversion. For the developers that would mean retaining the behaviour of the earlier versions, but only back as far as 3.0. For the users it would mean that once they had updated their files to 3.0 they could continue to use them without further modification if they wished. If the backward compatibilty subsequently became a burden then the developers could start again with version 4.0 and write a convert-ly just from 3.x to 4.0. Colin Fairchild wrote: LilyPonders - Users dealing with LilyPond as it evolves are presented with difficult choices, none good. If one knows a score will be completed quickly, never be revisited, and components of the code structure never will be reused, the choice is easy: save and share graphics only, dump the ly files for completed scores, and move on to the latest stable version. I'm not that prescient. Constantly adopting the latest version allows use of the latest features, feedback helps the developers, and the developers provide support. However, modifying 'finished' scores to be acceptable by the latest version is not reasonable. Upgrade modifications require significant effort. The convert-ly program helps, but misses a lot. It is tempting to select a stable version and stick with it. Scores can be revisited easily. Syntax and semantics are stable. Downside: the feature set never gets better and support will fade away unless a sufficient number of users make the same choice and help each other. It is possible to maintain multiple LilyPond versions. This allows revisiting old scores, but at a price. The operational environment becomes more and more confusing as versions accumulate. I have coded tens of scores, most with version 2.4.6, and spent several hours attempting to move just one (a smaller one) of them to 2.8.5. After using convert-ly and correcting for its errors, much remains to be done, especially to position the right text font. Conclusion: repetitive conversion of scores is untenable. The only reasonable solution is to maintain upward compatibility in the LilyPond processor. New features should be added without changing existing syntax. If it is deemed absolutely necessary to change semantics or define conflicting syntax, provide for optional interpretations based on the version specified. Older ly files should generate consistent results as LilyPond migrates. More exhaustive regression tests are necessary. If you can identify a better way, or have other comments, please respond. - Bruce ___ lilypond-user mailing list lilypond-user
Re: Evolutionary User Strategy - A Compromise
On Wednesday 12 July 2006 12:00, Colin Wilding wrote: This is an important dilemma for many users, I think - we want to have all the fixes and features in each new version, but find it frustrating when music produced in earlier versions needs time-consuming manual editing to upgrade. Can I suggest a compromise? I accept that Lilypond has been evolving rapidly and feel privileged to have been able to use it (and contribute comments) during that evolution. At some point, though, the evolution should slow down: the developers should feel happy with the basic structure and syntax and the users should be able to expect that music written for today?s version will look the same in tomorrow?s. What do you mean by 'the same'? Is it OK if the output suddenly looks better because a bug has been fixed? If yes, then you still have the problem that an upgrade may break your tweaks that work around layout bugs. If no, then development will be very difficult because all bugs have to be preserved. If there's sufficient community interest in maintaining and developing a conservative version of lilypond, then anyone can make a fork; however, I doubt any of the current developers would join. Lily is currently at a pretty early stage of development IMHO. How about making a resolution that from version 3.0 onwards Lilypond will be backwards-compatible: in other words, the current version will correctly display a file written in any earlier version 3.x without the need for conversion. I think 2.4 files are fairly forward compatible; there have been no major changes that I know of. There's also the question of what you mean by compatibility: Very advanced tweaks usually rely on the way lily's internals are organised, which may change over time. Since lily contains a Turing-complete programming language, for some language updates it is thereby _impossible_ to create a script that upgrades _all_ .ly files perfectly. There are also deprecated features that are dropped sometimes, this causes a kind of backward incompatibility. -- Erik ___ lilypond-user mailing list lilypond-user@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-user
Re: Evolutionary User Strategy - A Compromise
On Wednesday 12 July 2006 12:59, Anthony Youngman wrote: Something I thought of (having seen the comment about convert-ly using grep ...) It's not using grep, but it relies heavily on regexps (so it can somewhat fairly be compared to sed). I've got an on-off thing about writing a DATABASIC compiler (never mind) and have come across a tool called Antlr. It is a compiler-compiler and generates lexers, parsers and treeparsers. IF someone wants to put the effort in, it may (or may not be) easy to define various grammars to read in and chuck out different music formats. It sounds as though a lot of the problems (like the swap between and for example) would be easy. Especially given lilypond's structure it looks like it would be fairly easy to define grammars which can read in or chuck out different lily version syntaxes, even the \addLyrics / \oldAddLyrics thing maybe. I'd rather have a look at parsing expression grammars, see: http://en.wikipedia.org/wiki/Parsing_expression_grammar Seems to combine simplicity of regexps with the power of CFGs. Doesn't seem to be widely implemented though. Another way could be to go via the syntax expressions I'm working on: Instead of applying operations on .ly input, we could apply them to the parser's output, and then use some built-in version specific mechanism to convert finished files back to .ly. I thikn this would be both more reliable than convert-ly, and possibly easier to maintain, but some text formatting might be lost unless the syntax-expression-ly conversion is designed very carefully. Depending on how macros will be implemented, commands like \relative may be lost as well. -- Erik ___ lilypond-user mailing list lilypond-user@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-user
Re: Evolutionary User Strategy - A Compromise
On 7/12/06, Erik Sandberg [EMAIL PROTECTED] wrote: On Wednesday 12 July 2006 12:59, Anthony Youngman wrote: Something I thought of (having seen the comment about convert-ly using grep ...)It's not using grep, but it relies heavily on regexps (so it can somewhat fairly be compared to sed). I've got an on-off thing about writing a DATABASIC compiler (never mind) and have come across a tool called Antlr. It is a compiler-compiler and generates lexers, parsers and treeparsers. IF someone wants to put the effort in, it may (or may not be) easy to define various grammars to read in and chuck out different music formats. It sounds as though a lot of the problems (like the swap between and for example) would be easy. Especially given lilypond's structure it looks like it would be fairly easy to define grammars which can read in or chuck out different lily version syntaxes, even the \addLyrics / \oldAddLyrics thing maybe.I'd rather have a look at parsing _expression_ grammars, see:http://en.wikipedia.org/wiki/Parsing_expression_grammar Seems to combine simplicity of regexps with the power of CFGs. Doesn't seem tobe widely implemented though.another possibility might be to upgrade convert-ly to use something like pyparsing ( http://http://pyparsing.wikispaces.com/). Still, you can't make a 100% perfect converter../S ___ lilypond-user mailing list lilypond-user@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-user
RE: Evolutionary User Strategy - A Compromise
I don't really understand grammars etc (which is why my DATABASIC thing is on/off :-). But from my experience with Antlr, I don't see why you should lose stuff. Your PEG article mentions ASTs. I don't see that converting a .ly file into an AST can be that hard. So, for example, we write a Antlr grammar that creates a lexer/parser that turns the .ly into an AST. We now write another grammar that converts the AST to a .ly file. Provided we freeze the AST layout, it's dead easy to handle things like the / switch - the input grammar reads and the output grammar writes etc. The \addLyrics problem is a bit harder. I don't know whether we could parse both old and new versions into the same AST, but if we couldn't, a treeparser to convert between ASTs isn't that hard. Antlr does things like converting between Java and C# ... !!! Anyways, I would expect that we could share one AST between several incompatible .ly variants of lily, and within that, an either direction transform would be easy. It also means that adding import (or export) of other formats would be easy - to import, write a parser grammar to read the other format and a treeparser to transform the resulting AST to the lily version. For export, go the other way. But it modularises everything. And you wouldn't lose things like \relative because you're using a pure lexer/parser setup, that wouldn't even understand what \relative was :-) I might play with this when I get my hands on Antlr 3, but don't bank on it. Cheers, Wol -Original Message- From: Erik Sandberg [mailto:[EMAIL PROTECTED] Sent: 12 July 2006 13:55 To: lilypond-user@gnu.org Cc: Anthony Youngman Subject: Re: Evolutionary User Strategy - A Compromise On Wednesday 12 July 2006 12:59, Anthony Youngman wrote: Something I thought of (having seen the comment about convert-ly using grep ...) It's not using grep, but it relies heavily on regexps (so it can somewhat fairly be compared to sed). I've got an on-off thing about writing a DATABASIC compiler (never mind) and have come across a tool called Antlr. It is a compiler-compiler and generates lexers, parsers and treeparsers. IF someone wants to put the effort in, it may (or may not be) easy to define various grammars to read in and chuck out different music formats. It sounds as though a lot of the problems (like the swap between and for example) would be easy. Especially given lilypond's structure it looks like it would be fairly easy to define grammars which can read in or chuck out different lily version syntaxes, even the \addLyrics / \oldAddLyrics thing maybe. I'd rather have a look at parsing expression grammars, see: http://en.wikipedia.org/wiki/Parsing_expression_grammar Seems to combine simplicity of regexps with the power of CFGs. Doesn't seem to be widely implemented though. Another way could be to go via the syntax expressions I'm working on: Instead of applying operations on .ly input, we could apply them to the parser's output, and then use some built-in version specific mechanism to convert finished files back to .ly. I thikn this would be both more reliable than convert-ly, and possibly easier to maintain, but some text formatting might be lost unless the syntax-expression-ly conversion is designed very carefully. Depending on how macros will be implemented, commands like \relative may be lost as well. -- Erik * * This transmission is intended for the named recipient only. It may contain private and confidential information. If this has come to you in error you must not act on anything disclosed in it, nor must you copy it, modify it, disseminate it in any way, or show it to anyone. Please e-mail the sender to inform us of the transmission error or telephone ECA International immediately and delete the e-mail from your information system. Telephone numbers for ECA International offices are: Sydney +61 (0)2 8272 5300, Hong Kong + 852 2121 2388, London +44 (0)20 7351 5000 and New York +1 212 582 2333. * * ___ lilypond-user mailing list lilypond-user@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-user
Re: Evolutionary User Strategy - A Compromise
On Wednesday 12 July 2006 17:22, Anthony Youngman wrote: I don't really understand grammars etc (which is why my DATABASIC thing is on/off :-). But from my experience with Antlr, I don't see why you should lose stuff. Your PEG article mentions ASTs. I don't see that converting a .ly file into an AST can be that hard. So, for example, we write a Antlr grammar that creates a lexer/parser that turns the .ly into an AST. We now write another grammar that converts the AST to a .ly file. A problem here is code duplication; it takes some effort to maintain two parsers instead of one. I think it will be difficult to automatically test that the current antlr parser corresponds well with the actual grammar the current lilypond version uses. I have been thinking about moving lily's entire parser out to Scheme; this way we could keep one old parser for each version, and use it to generate an AST, which then is converted nicely using rules written in Scheme (Scheme rocks when it comes to tree manipulation). I'm not sure if it's possible though. BTW: Will your solution handle whitespace nicely? -- Erik ___ lilypond-user mailing list lilypond-user@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-user
Re: Evolutionary User Strategy - A Compromise
Erik Sandberg wrote: There's also the question of what you mean by compatibility: Very advanced tweaks usually rely on the way lily's internals are organised, which may change over time. Since lily contains a Turing-complete programming language, for some language updates it is thereby _impossible_ to create a script that upgrades _all_ .ly files perfectly. Actually, doesn't this mean that we can _always_ upgrade _all_ .ly files perfectly? Since lily contains a Turing-complete language, we just need to write a universal Turing machine which emulates the behavior of 2.4 (or whatever). Then convert-ly takes 2.4, adds our Turing machine, and presto, we have a .ly file which compiles under 2.8. Of course, that .ly file is probably about 20 megabytes in size, and would take a huge amount of time to run... but it would work! :) Cheers, - Graham ___ lilypond-user mailing list lilypond-user@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-user
