Re: New alist to replace special characters. (issue4553056)
lily/text-interface.cc updated. http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
I updated the patch. There's now a list of special characters and a \replace command for markups. The escape character is now '§'. It's the only one that works great with lyrics. And it isn't used elsewhere in LilyPond's syntax. The syntax for using the list of special characters #(include-special-characters) is a bit ackward but I didn't found anything better. Regards, Bertrand http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
passes make and reg tests http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
On 7/31/11 11:41 AM, bordage.bertr...@gmail.com bordage.bertr...@gmail.com wrote: I updated the patch. There's now a list of special characters and a \replace command for markups. The escape character is now '§'. It's the only one that works great with lyrics. And it isn't used elsewhere in LilyPond's syntax. This escape is not available on US keyboards, so it's as much work to use the escape character as to put in the UTF character directly. For me, as a US user, that seems to negate the purpose for the patch. What was the problem with the previous escape characters? Thanks, Carl ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
Ouch...
The problem with '' is that it fails on lyrics:
this works:
\new Lyrics \lyricmode { as; }
but this doesn't:
\new Lyrics \lyricmode { s; }
There is the same kind of issues with almost every easy-to-type special
character:
@ % $ # \ / ^ ~ + = * ; ( ) [ ] { }
This is the exact list of every possible escape characters for a french
keyboard under linux:
§ £ µ € ¤ ¹ ² ' ` © ¨ ø Ø ≤ ≥ « » “ ” ↓ ¬ × ÷ ¿ ¡ ∕ ⋅ …
Of course, only a few of them are available under windows:
§ £ µ € ¤ ' `
If we consider US, british, german and spanish keyboards, this only
gives these two very bad characters:
' `
The best solution may be to solve this lyric problem first :\
Bertrand
http://codereview.appspot.com/4553056/
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
Patch updated. I abandonned the idea of including a special characters list. http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
LGTM. You could add a .ly file that defined a more comprehensive set of special characters as an optional item that the user includes with \include. That would allow its use without requiring its use. THanks, Carl http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
A new file in ly with definitions using add-text-replacements!?
Like this?
germanHTML = #(add-text-replacements! '((ss; . ß)
[...])
englishLaTeX = [...]
This way, users can easily stack special characters with this syntax:
\paper {
\englishHTML
\frenchLigatures
}
Besides, I made some performance tests.
With a big replacement alist, building Nicolas Sceaux's huge Atys takes
0.7s more than without it.
Thanks,
Bertrand
http://codereview.appspot.com/4553056/
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
passes make and reg tests http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
http://codereview.appspot.com/4553056/diff/9003/input/regression/markup-special-characters.ly File input/regression/markup-special-characters.ly (right): http://codereview.appspot.com/4553056/diff/9003/input/regression/markup-special-characters.ly#newcode7 input/regression/markup-special-characters.ly:7: #(define-markup-list-command (show-special-characters layout props) () It was a way to remember this piece of code. I will not do all the documentation before deciding if the list remains. Anyway, there are other examples of such lists in regression testing. For example, 'chord-name-entry' or 'woodwind-diagrams-empty'. http://codereview.appspot.com/4553056/diff/9003/scm/output-lib.scm File scm/output-lib.scm (right): http://codereview.appspot.com/4553056/diff/9003/scm/output-lib.scm#newcode958 scm/output-lib.scm:958: ; Punctuation I understand your concerns and I'm a little embarrassed to introduce a new syntax for the risk of disrupting the users understanding. But it also lacks a lot of people. In the list, I added the characters which are not allocated in the French Linux keyboard (except the ellipsis, the non-breaking space and the dashes). It does bother me however to remove this list. http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
http://codereview.appspot.com/4553056/diff/9003/scm/output-lib.scm File scm/output-lib.scm (right): http://codereview.appspot.com/4553056/diff/9003/scm/output-lib.scm#newcode958 scm/output-lib.scm:958: ; Punctuation On 2011/06/07 09:45:12, Bertrand Bordage wrote: I understand your concerns and I'm a little embarrassed to introduce a new syntax for the risk of disrupting the users understanding. But it also lacks a lot of people. In the list, I added the characters which are not allocated in the French Linux keyboard (except the ellipsis, the non-breaking space and the dashes). It does bother me however to remove this list. It may be worth testing this out on multiple platforms - I've sometimes had difficulty opening up my .ly files on Windows machines if they contain special characters. To what extent is UTF-8 easy to work with outside of a posix environment? In general, one way to work this into LilyPond may be to include the basic functionality, send the string-replacement-alist out to lilypond-user, and see to what extent people desire to use it. I've done this for a couple things and been able to scrap/add functionalities accordingly. If it seems like it's something that many people would want to use, then it should probably be part of the distro. http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
A replacement function for text is a very good idea, and would be very useful. General comments: A regtest for lyrics might be missing. The current implementation of replacement function, in the C++ part, need some rework. An alternate design might be: Define in the guile part the replacement function. The C++ code would just call that function on the string. By default, the replacement function could do exactly the job that is done in current LilyPond version (replacing line feeds, etc, by spaces). Another replacement function would be proposed, doing the work you propose here, e.g. compiling and applying a regexp, instead of manually implementing the replacement (in a possibly less efficient way than the built-in would do). And a \useTheFancyTextReplacementThing switch could select the later replacement function. (just brainstorming) If the second replacement function has not much impact on performances, it may even be the default. http://codereview.appspot.com/4553056/diff/9003/input/regression/markup-special-characters-shorthands.ly File input/regression/markup-special-characters-shorthands.ly (right): http://codereview.appspot.com/4553056/diff/9003/input/regression/markup-special-characters-shorthands.ly#newcode15 input/regression/markup-special-characters-shorthands.ly:15: (additional-replacements text-font-defaults The interface for adding replacements/shortcuts seems a bit complicated. Maybe a function call modifying text-font-defaults behind the scene would be better? #(add-text-replacements! '(...)) http://codereview.appspot.com/4553056/diff/9003/lily/text-interface.cc File lily/text-interface.cc (right): http://codereview.appspot.com/4553056/diff/9003/lily/text-interface.cc#newcode46 lily/text-interface.cc:46: for (int j = 0; j scm_to_int (scm_length (replacement_alist)); j++) This is not the way to loop over a list: the idiom can be found many times in the C++ code. scm_length and scm_list_ref are O(n). What's more, at this point the `str' argument is not yet used. So maybe this could be cached to avoid being computed again each time. http://codereview.appspot.com/4553056/diff/9003/lily/text-interface.cc#newcode62 lily/text-interface.cc:62: str-replace (i, j, ligature); We see three embedded loops: while, for, and a call to ly_assoc_get... The previous add only one loop. What is the impact on performance, wrt to current version? Have you considered compiling (and caching) a regexp and use the appropriate builtin function which will do all the replacement work? Or maybe... get this remplacement part out of C++ code, and define in the scheme part a replacement function, that will be called here. A default replacement function might do the \n, \t \v replacement alone (as on current version). An other replacement function would be proposed which does more complicated things. http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
Done ! Regression tests are ok. http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
Patch LGTM. I still have concerns about maintaining a list of ascii definitions for UTF-8 characters, as described in the comments. http://codereview.appspot.com/4553056/diff/9003/input/regression/markup-special-characters.ly File input/regression/markup-special-characters.ly (right): http://codereview.appspot.com/4553056/diff/9003/input/regression/markup-special-characters.ly#newcode7 input/regression/markup-special-characters.ly:7: #(define-markup-list-command (show-special-characters layout props) () If we're going to do this, this code should be added to the appendices of the Notation Reference, so it will be included in the documentation. I don't really think it should be part of the regression test suite, but I could be convinced otherwise. http://codereview.appspot.com/4553056/diff/9003/scm/output-lib.scm File scm/output-lib.scm (right): http://codereview.appspot.com/4553056/diff/9003/scm/output-lib.scm#newcode958 scm/output-lib.scm:958: ; Punctuation I'm still not convinced that we should get in the business of defining these ascii-based characters instead of using UTF-8 directly as part of the core lilypond distribution. But if nobody else is opposed to it then I will not maintain my objection and I will agree with the patch. I have *no* objections to including the functionality with whitespace defined in the default-string-replacement-alist, and having an LSR snippet with additional replacements. http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
Ok, I'll change these. What about 'No.' ? 'numero;', 'N°' or 'N°;' ? http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
New patch set with Carl's ideas. I don't know if replacement-alist should be defined by default. Nor if '' is a good escape character. I chose it because it's quicker to type on a french keyboard than '@'. Bertrand. http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
Philosophically, utf-8 is the One True Way, but I am sympathetic to laxy typists. On 2011/06/03 11:11:57, Bertrand Bordage wrote: I don't know if replacement-alist should be defined by default. The shorter the default list, the fewer surprises. The keys should not include any possibly-desired input. Nor if '' is a good escape character. It is reasonable, but draw from an existing string-replacement set like HTML and use the closing ;. Then we will be less-likely confused when I reference as records (or when you reference rs records) in a markup on a score. http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
http://codereview.appspot.com/4553056/diff/5002/scm/output-lib.scm File scm/output-lib.scm (right): http://codereview.appspot.com/4553056/diff/5002/scm/output-lib.scm#newcode959 scm/output-lib.scm:959: (... . …) (hellip; . …) (emdash; . –) etc. http://codereview.appspot.com/4553056/diff/5002/scm/output-lib.scm#newcode974 scm/output-lib.scm:974: (ordM . º) (ordm; . º) etc. http://codereview.appspot.com/4553056/diff/5002/scm/output-lib.scm#newcode979 scm/output-lib.scm:979: (s . ſ))) HTML just uses the unicode number #x17F; for long s, so maybe just s; http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
New alist to replace special characters. (issue4553056)
Reviewers: , Message: A feature that helps text typesetting. Bertrand Description: New alist to replace special characters. Please review this at http://codereview.appspot.com/4553056/ Affected files: A input/regression/markup-special-characters-shorthands.ly M lily/text-interface.cc M ly/paper-defaults-init.ly A ly/special-characters.ly M scm/define-grob-properties.scm ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
This prints every character with its shorthand, but works bad because of '\t', '\v' and '\n' : \version 2.15.0 \include special-characters.ly #(define-markup-list-command (show-special-characters layout props) () (interpret-markup-list layout props (map (lambda (pair) (markup #:fill-line ( #:null #:override '(replacement-string-max-length . 0) (car pair) (car pair) #:null))) special-characters-alist))) \markuplines \show-special-characters http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
On May 23, 2011, at 12:38 PM, bordage.bertr...@gmail.com wrote: This prints every character with its shorthand, but works bad because of '\t', '\v' and '\n' : \version 2.15.0 \include special-characters.ly #(define-markup-list-command (show-special-characters layout props) () (interpret-markup-list layout props (map (lambda (pair) (markup #:fill-line ( #:null #:override '(replacement-string-max-length . 0) (car pair) (car pair) #:null))) special-characters-alist))) \markuplines \show-special-characters Good work! A couple things : (1) If I want to write \emdash is a command in lilypond, that won't work (I think) with this patch. You need to create either a ( . \\) special character or another workaround for the backslash. (2) For ligatures that arrive at the end of the word like in some German texts, you'll need to figure out where the ligature falls in the word. Writing a single exception to do this in C++ is easy, but if ligatures are any hairier than this (i.e. if in Nabakov's Zemblan there is an ie ligature except after c), then you'd be better off porting the whole thing to Scheme and doing it via regular expressions or some other sorta text replacement algorithm. The added advantage of this is that users can specify their own rules. (3) Newbs will invariably learn about this feature, want to use it, and then complain that the ligature they want is in a different font than Century Schoolbook in the PDF. Perhaps lilypond should issue a warning if a character is found in a font that is not the one you currently have selected like Warning: Charater `foo' found, but not in the current font. Cheers, MS ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
(1) You can do it using \override #'(replacement-string-max-length . 0) But I agree, there's a problem with backslashes. (2) What if we define something like \tz when we want 'ꜩ' and just write tz when no ligature is needed ? This isn' t so constraining. (3) I know, this is another change to be done. And when this patch will be pushed, I'll take time to add a doc entry. Bertrand http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
http://codereview.appspot.com/4553056/diff/1/ly/special-characters.ly File ly/special-characters.ly (right): http://codereview.appspot.com/4553056/diff/1/ly/special-characters.ly#newcode1 ly/special-characters.ly:1: #(define special-characters-alist Is there any (public) command to add a replacement to that alist (or remove one)? Or do you have to repeat the whole list if you want to add/remove just one entry? http://codereview.appspot.com/4553056/diff/1/ly/special-characters.ly#newcode14 ly/special-characters.ly:14: (\nbhyphen . ‑) \\nbhyphen? http://codereview.appspot.com/4553056/diff/1/ly/special-characters.ly#newcode36 ly/special-characters.ly:36: (_ . ) ; thinspace Will this work with words that have a wanted underscore? Or will the underscore always be replaced with a thinspace? http://codereview.appspot.com/4553056/diff/1/ly/special-characters.ly#newcode41 ly/special-characters.ly:41: (\textbullet . •) \\textbullet ? http://codereview.appspot.com/4553056/diff/1/ly/special-characters.ly#newcode80 ly/special-characters.ly:80: (\ng . ŋ) Should that be \\ng? http://codereview.appspot.com/4553056/diff/1/ly/special-characters.ly#newcode85 ly/special-characters.ly:85: (\\OE . Œ) Is the replacement always done on the longest string found? In particular, if you wan to write ØE (Øe is a danish name apparently), you'll have to use \\OE, which might or might not be matched with the sequence for Œ instead... http://codereview.appspot.com/4553056/diff/1/ly/special-characters.ly#newcode88 ly/special-characters.ly:88: (\th . þ) \\th? http://codereview.appspot.com/4553056/diff/1/ly/special-characters.ly#newcode94 ly/special-characters.ly:94: (\times . ×) \\times http://codereview.appspot.com/4553056/diff/1/ly/special-characters.ly#newcode97 ly/special-characters.ly:97: (\twosuperior . ²) \\twosupererior http://codereview.appspot.com/4553056/diff/1/ly/special-characters.ly#newcode98 ly/special-characters.ly:98: (\threesuperior . ³) \\threesuperior ? http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
http://codereview.appspot.com/4553056/diff/1/lily/text-interface.cc File lily/text-interface.cc (right): http://codereview.appspot.com/4553056/diff/1/lily/text-interface.cc#newcode40 lily/text-interface.cc:40: int max_length = scm_to_int (ly_chain_assoc_get (ly_symbol2scm (replacement-string-max-length), Why is string-max-length needed? http://codereview.appspot.com/4553056/diff/1/ly/special-characters.ly File ly/special-characters.ly (right): http://codereview.appspot.com/4553056/diff/1/ly/special-characters.ly#newcode1 ly/special-characters.ly:1: #(define special-characters-alist I think this should be a .scm file, rather than a .ly file. I'm concerned about the default position of moving from UTF-* to ascii for special characters. I think that's moving in the wrong direction. I do think the ligature replacement is very good. But I could easily be persuaded that this is the right thing to do. http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
On 5/23/11 3:32 PM, bordage.bertr...@gmail.com bordage.bertr...@gmail.com wrote: http://codereview.appspot.com/4553056/diff/1/lily/text-interface.cc#newcode40 lily/text-interface.cc:40: int max_length = scm_to_int (ly_chain_assoc_get (ly_symbol2scm (replacement-string-max-length), It is needed line 50. This can be inferred from replacement-alist. But with this property we can disable every shorthand or just those longer than replacement-string-max-length. The ideas of disabling shorthands by the length of the replacement string seems very hackish to me. It seems unlikely to me that I would ever decide which replacement strings to disable by the length. To disable all replacement strings, we should have something like the following functionality #(define default-string-replacement-alist '(; This would be your list )) to enable replacements: \override TextScript #'replacement-alist = #'default-string-replacement-alist to disable replacements: \override TextScript #'replacement-alist = #'()_ @Carl : I agree for the .scm file. Why do you think it is going in the wrong direction ? Because we are making up our own syntax for characters that are already defined as UTF characters. We should avoid making up syntax if at all possible. For the ligatures, I'm supportive; we don't have a text processor that will automatically handle ligatures. For the special characters like trademark, and super2, we should just have them inserted into the file as UTF characters, IMO. I don't think we should be trying to (for example) handle TeX input in LilyPond strings. I see that as a maintenance pain in the neck. We're moving in the direction (if we're not there already) of correctly handling UTF text. Why not just let the person who wants to use UTF text do so? What is the advantage to typing \\aa instead of å? Thanks, Carl ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
Yes. Knowing this, I suggest we keep whitespaces, punctuation, quotes and word dividers (with some small changes). There's still something that bothers me : isn't there some special characters that you can't do with you keyboard ? Even on linux I can't type some symbols like ſ or • without copying/pasting. So, to your question What is the advantage to typing \\aa instead of å? I answer I don't have any key for this. Cheers, Bertrand http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
On 2011/05/23 22:15:38, Bertrand Bordage wrote: Yes. Knowing this, I suggest we keep whitespaces, punctuation, quotes and word dividers (with some small changes). There's still something that bothers me : isn't there some special characters that you can't do with you keyboard ? Even on linux I can't type some symbols like ſ or • without copying/pasting. So, to your question What is the advantage to typing \\aa instead of å? I answer I don't have any key for this. See the Wikipedia entry on Unicode input for ways to get characters that aren't on your keyboard: http://en.wikipedia.org/wiki/Unicode_input HTH, Carl http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
...definitely not user-friendly! I totally agree it's better to type in UTF and that's what I always did with LilyPond. But this REALLY wastes time when we type several symbols and ligatures. Making this easier should be the OS's job. I'll think more about this and make another patch set. Bertrand http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: New alist to replace special characters. (issue4553056)
On 2011/05/23 23:11:33, Bertrand Bordage wrote: Making this easier should be the OS's job. Yes, I agree. That's why I'm not in favor of making it part of LilyPond for us to maintain. Having the facility to do the general substitution as part of LilyPond is fine, IMO. Having the list of all the Unicode characters is not a good idea. Having that list as an LSR snippet would be OK. That way it's not part of the contract we have with users to maintain it. Thanks, Carl http://codereview.appspot.com/4553056/ ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
