On Wed, Sep 16, 2015 at 09:02:23PM +0000, Guenter Milde wrote: > Dear Enrico, > > On 2015-09-14, Enrico Forestieri wrote: > > On Mon, Sep 14, 2015 at 12:52:34PM +0000, Guenter Milde wrote: > > >> >> A complete discussion and patch are at > >> >> http://www.lyx.org/trac/ticket/9742 > > >> > I think that all is needed is the attached patch. > > >> While the patch fixes the worst case (i.e. the above example), it is no > >> solution for #9742: > > >> * open LyX, > >> * open a new document, > >> * open a math box (CTRL-M) > >> * write \AA > > >> The ERT is replaced by a blue italic Å, indicating that LyX regards "\AA" > >> as > >> a valid math command. > > >> However, when compiling, there is the error > > >> Missing character: There is no � in font cmr10! > > ... > > > This is a more general problem that is difficult to get right in all > > circumstances. > > I hope we can agree on the expected behaviour in mathed under "normal" > circumstances: > > * If a command is suggested as autocompletion and/or "recognized" via > replacement by a graphical representation, this command should work as > indicated. > > * If a command is unknown (or known as problematic), it should stay red. > > Of course, there are the known limits: > > a) The custom LaTeX preamble is not parsed. Packages loaded in the user > preamble may clash with other packages or change commands. The same > holds for command definitions and re-definitions that LyX cannot take > care of. > > b) ERT is not parsed by LyX, so LyX cannot ensure things work. > > Problematic code in the custom LaTeX preamble or ERT is what I'd call > "exceptional" circumstances, where the user handles on his/her own risk. > > > Bug #9742 concerns 20 text-mode commands where LyX gets this wrong under > "normal" circumstances: text-only commands that LyX displays as > "recognized math commands". This is a specific problem that can be led > back to these 20 definitions in lib/symbols. > > Removing these definitions > > > > The lib/unicodesymbols replacements are not a panacea > > that always work, and sometimes those replacements simply get in the way. > > OK. If this happens under normal circumstances, this is a bug and needs > handling. Under exceptional circumstances, these problems are "wontfix" bugs. > > > As an example I attach here a file where I introduced a literal Å in > > mathed and the lib/unicodesymbols replacement is wrong (try compiling). > > This example is an instance of exceptional circumstances: > > a user preamble defining a mode-changing command > and use of this command in mathed > > And indeed, LyX's Unicode replacement mechanism is (unsurprisingly) > confused. > > This is also the case the other way round: in your example replace the "Å" > with "x^2". The command will turn blue ("recognition as math command") but > compilation will fail: > > ! Missing $ inserted. > > I am not going to argue that this is a problem with mathed, just that > "unicodesymbols" in mathed is not worse than the rest of mathed. > > However, the example also shows how LyX should (and in this case does) > handle exceptional cases: the "homegrown" command in the math box stays > red: inidicating that this is unsafe content. > > ... > > > > I think that it is very difficult, if not impossible, catching all quirks > > of latex. IMO lyx should simply help in the most common cases and not trying > > to be too smart... This does not > > mean that one should not try to address an issue, but I rather avoid radical > > changes. Those macros had been defined in the symbols file for a very long > > time > > They may have been usefull at this time, when LyX did not support Unicode. > Definitions for \AA, \O, and \textdegree were required to render the symbols > if inserted via the "misc" toolbox but this can be implemented via Unicode > characters now (see my patch). > > > ... and I find them useful. > > What is your use case? Does it involve the "Extended Latin" chars or the > wasy symbols? > > Writing \O in mathed resulted in a Ø in the GUI but a promille sign > in the output - whether in math-mode or in text-in-math mode! A \phone > becomes an \Upsilon if input in math-mode. I don't think this feature was > much used - there was no bug report in all this time. > > > > You pointed out a problem and a partial solution > > was found. > > I did not have the time to test the second patch but I suppose that it > solves the problem of uncompilable documents. However, I still think that > text-mode command definitions in lib/symbols are conceptually wrong: > > * code duplication (command <-> symbol mappings in both, lib/symbols and > lib/unicodesymbols). > > * about 20 text commands are shown among the autocompletions for math > commands. This obfuscates the important distinction between text mode and > math mode in LaTeX. (For me, LyX is also a valuable LaTeX learning tool.) > > * Writing \phone in math mode would now result in \textit{\phone}. However, > old documents with wrong use would still show the phone symbol in the GUI > but an Y in the output (without error or warning!). Even new documents > could be tricked into this. > > (Unless the display of the command as symbol is also made conditional on > the "textmode" flag.) > > > I don't think that my approach (removing text commands from lib/symbols) is > "radical". It would simplify the code base without rendering any old > document uncompilable. The non-supported cases (text-commands in math mode) > would simply be displayed as ERT in the GUI (properly indicating that this > is not supported) but the output would remain unchanged. -> no need for a > file version change. > > OTOH, a comprehensive support for LaTeX input of text symbols is almost > there: > > * There is Edit>Paste Special>Paste from LaTeX, > * LICR commands in text-in-math are converted from ERT into the correct > symbol after first save or save/reload, > * LICR commands are converted to the correct symbol when pasted into > text-in-math, > * LICR commands in text-in-math are converted when text is turned into math: > > In LyX > * type (as text) "\text\texteuro", > * select it > * press Ctrl-M > and see the Euro sign appear. > > What is missing: > > * instant conversion of a LaTeX command after typing, > * auto-completion. > * a way to achieve this also without wrapping in $\text{ }$. > > Maybe changes to the code would be smaller than your updated patch.
Please note that the patch you propose is not changing anything. You simply force the use of unicode symbols through the math panels. However, if a user directly inputs \AA this is now displayed in red, even if it is entered in a text mode environment. One has to save and reload the file to actually see that symbol. Moreover, note that saving and reloading also solves the problem occurring without your patch that those symbols are enclosed in \ensuremath [1]. Try it. So, your patch is rather a step backward. The patch I am proposing let you display those symbols even if they are entered through a latex command and assures that no compile error occurs even without the need of saving and reloading. Instead your patch causes errors even after saving and reloading (and still leaves in red the symbol). As regards the value as a learning tool, I don't see how this can be countered by the fact that instead of remaining red a symbol is enclosed in a mode changing environment. Indeed, a learning user will see in the source panel that his symbol needs something else. [1] From a technical point of view, this is because the macros defined in lib/symbols are considered as MathMacros, i.e., the same category of the macros that are input through the LyX interface. Now, the code assumes that all MathMacros need a math environment and thus they are enclosed in \ensuremath. This is a decision that was made at that time, and it cannot be changed because it would break existing documents. However, with the proposed patch attached to #9742 LyX is now instructed that some MathMacros actually are textmode macros and thus it is able to avoid compiling errors. -- Enrico
