Re: [PATCH] Add support for $…$ latex fragments followed by a dash
Rudolf Adamkovič writes: > João Pedro de Amorim Paula writes: > >> On 01 February 2022 22:00, Rudolf Adamkovič wrote: >> >>> Me, I cannot use any of these "pretty" features because, simply put, >>> they break plain text. For example, they cause misaligned tables and >>> make the text overflow the fill column, which I keep at 72 columns. >> >> […] there are fonts that enforce Unicode (all characters, from what I >> understand, Unicode included) characters to be exactly 1 unit width, >> i.e. Unicode characters are the same widths as other characters. At >> least I can guarantee that any of the Term (all characters are the >> same length, but has ligatures) or Fixed (same as Term but no >> ligatures) for Iosekva [1] have this given property. >> >> [1] https://typeof.net/Iosevka/ > > Thank you for sharing! I use the amazing "Hack" typeface [1]. Then, I > fail to understand how the font changes the fact that Org breaks "the > promise of plain text" in this regard. > > For example, Org can hide its emphasis markers. Later, one opens the > file in another editor and sees the truth. The lines go over the fill > column, the tables have misaligned columns, and so on. > The promise of plain text has no inherent promise regarding alignment, line wrapping etc. The promise of plain text is simply that - a promise that org files will be just plain text rather than some application specific format like many other systems (MS Word, LibreOffice, etc). Provided you can still edit the file with a plain text editor, org has not broken its promise. Ironically, it is this plain text basis of org files which makes on-going support of $..$ (and any extension) so problematic. Without this restriction, org files could have a format where elements were 'tagged' to remove ambiguity and simplify the parsing process. However, this would of course mean that either you had to edit the file within org mode or you would have to be intimately familiar with the internal structure of org mode files and use an editor which made editing/updating the internal structure possible. To be very clear, I am NOT advocating that org should adopt some form of internal tagging or structure. I'm only attempting to highlight that having a system which aims to maintain the data source in plain text comes at a cost. In LaTex et. al. $..$ works well because $ is a special character. If you want a literal $, you have to escape it. In org, $ is not a special character. This means we need complex regular expressions in order to identify when $ is being used for LaTex specific markup and when it is being used in other contexts (e.g. literal $ sign, variable name). These regular expressions are difficult to get right, error prone and above all, incur a significant performance hit. Extending the syntax further to support $..$- simply makes the situation worse by adding further complexity. So far, all the arguments against removal of support for $..$ are based on inconvenience and reduced readability associated with the alternatives. Both legitimate concerns. For many, the proposed change may seem to be just inconvenient change for no real purpose because they don't deal with the complexities inherent in making $..$ work and for others, change is something they would always prefer to avoid. In this instance, I feel we really need to listen to those who put in such a dedicated effort in maintaining org mode and accept their position that sustaining $..$ syntax going forward is problematic and focus on what can be done to mitigate the impact from this change, make the alternative syntax more convenient and address the readability issues.
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
João Pedro de Amorim Paula writes: > On 01 February 2022 22:00, Rudolf Adamkovič wrote: > >> Me, I cannot use any of these "pretty" features because, simply put, >> they break plain text. For example, they cause misaligned tables and >> make the text overflow the fill column, which I keep at 72 columns. > > […] there are fonts that enforce Unicode (all characters, from what I > understand, Unicode included) characters to be exactly 1 unit width, > i.e. Unicode characters are the same widths as other characters. At > least I can guarantee that any of the Term (all characters are the > same length, but has ligatures) or Fixed (same as Term but no > ligatures) for Iosekva [1] have this given property. > > [1] https://typeof.net/Iosevka/ Thank you for sharing! I use the amazing "Hack" typeface [1]. Then, I fail to understand how the font changes the fact that Org breaks "the promise of plain text" in this regard. For example, Org can hide its emphasis markers. Later, one opens the file in another editor and sees the truth. The lines go over the fill column, the tables have misaligned columns, and so on. Rudy [1] https://sourcefoundry.org/hack/ -- "Strange as it may sound, the power of mathematics rests on its evasion of all unnecessary thought and on its wonderful saving of mental operations." -- Ernst Mach, 1838-1916 Rudolf Adamkovič [he/him] Studenohorská 25 84103 Bratislava Slovakia
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
On 01 February 2022 22:00, Rudolf Adamkovič wrote: > Me, I cannot use any of these "pretty" features because, simply put, > they break plain text. For example, they cause misaligned tables and > make the text overflow the fill column, which I keep at 72 columns. I know that this is a bit divergent from the original subject -- and also keep in mind that this shouldn't be considered a solution to the problem, given that it should be considered for all users with different fonts --, but there are fonts that enforce Unicode (all characters, from what I understand, Unicode included) characters to be exactly 1 unit width, i.e. Unicode characters are the same widths as other characters. At least I can guarantee that any of the Term (all characters are the same length, but has ligatures) or Fixed (same as Term but no ligatures) for Iosekva [1] have this given property. [1] https://typeof.net/Iosevka/ Best regards, -- João Pedro de Amorim Paula IT undergraduate at Universidade Federal do Rio Grande do Norte (UFRN)
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
Ihor Radchenko writes: > We can theoretically make a change to support "-", but then it will be > logical to support $i$th as well. […] I do not know about others, but supporting the dash alone would mean a world to me. In fact, I never use \(\) for anything else in all my Org documents. That dash, though, it constantly gets into my way. > 1. Fontify \(...\) replacing the brackets with a single character. For >example: > > \(...\) -> ⁅...⁆ Me, I cannot use any of these "pretty" features because, simply put, they break plain text. For example, they cause misaligned tables and make the text overflow the fill column, which I keep at 72 columns. > 2. Provide convenient way to input \(\) brackets through >electric-pair-mode or trough org-cdlatex-mode. This would help a bit. Rudy -- "'Contrariwise,' continued Tweedledee, 'if it was so, it might be; and if it were so, it would be; but as it isn't, it ain't. That's logic.'" -- Lewis Carroll, Through the Looking Glass, 1871/1872 Rudolf Adamkovič [he/him] Studenohorská 25 84103 Bratislava Slovakia
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
manual states: Text within the usual LaTeX math delimiters. To avoid conflicts with currency specifications, single ‘$’ characters are only recognized as math delimiters if the enclosed text contains at most two line breaks, is directly attached to the ‘$’ characters with no whitespace in between, and if the closing ‘$’ is followed by whitespace, punctuation or a dash. For the other delimiters, there is no such restriction, so ^^ when in doubt, use ‘\(...\)’ as inline math delimiters. It is even more interesting. Support of dash likely was unintentionally lost in the following commit: https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=6779f8f424883ffd202e24cbd8bb4e241db294b0 that generalizes handling of punctuation, unfortunately dash and apostrophe are not always belong to punctuation symbols in *text* modes. That commit even updates manual to a less precise phrase, however .texi file only, so the change was lost. Nicolas later restored apostrophe in the commit c0369a798470763f8f3c69cf2079c3a194635feb False positive with the proposed patch: > Balance decreased from $10 to negative value ($-2 approximately) certainly it is more rare than $n$-th valid case. Tim, as mentioned before I’m strongly in favour of a ~half decade transition Half of decade already passed since dash after currency symbol was broken so maybe it is better to fix current state by updating the manual (including bugfix branch) and by adding some tests. P.S. It is deja vu, I almost certainly saw quite recent discussions whether punctuation may be handled in some regexps in more generic way. Consequences may be similar in respect to characters that are almost punctuation...
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
On 28/01/2022 23:37, Timothy wrote: There is no point to discuss deprecation of $…$ and ... while usage of such constructs is not discouraged by the org manual Isn’t the point of discussing depreciation now that we’d change the manual etc. to discourage it? 😛 It is funny that nobody have checked what the manual actually states: Text within the usual LaTeX math delimiters. To avoid conflicts with currency specifications, single ‘$’ characters are only recognized as math delimiters if the enclosed text contains at most two line breaks, is directly attached to the ‘$’ characters with no whitespace in between, and if the closing ‘$’ is followed by whitespace, punctuation or a dash. For the other delimiters, there is no such restriction, so ^^ when in doubt, use ‘\(...\)’ as inline math delimiters. I have no idea whether it ever worked in such way. So either the manual must be updated or a fix must be committed. False positive with the proposed patch: > Balance decreased from $10 to negative value ($-2 approximately) certainly it is more rare than $n$-th valid case. If we do deprecate support for $…$, it might also be a good idea to see if we can add a utility function which would make it easier for people to migrate existing documents to the new/alternative syntax. For the same reason it is hard to reliably parse $…$ syntax, we probably can’t automate that transition, but we should be able to reduce the effort required to update existing documents. Tim, as mentioned before I’m strongly in favour of a ~half decade transition including utility functions to shift existing TeX constructs to LaTeX, and adding warnings, well before dropping support. That period should be started when manual is updated. Functions updating old file to the current syntax is an interesting idea, but it assumes a copy of org-element as it was in that old org version.
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
Hi Max, > There is no point to discuss deprecation of $…$ and ... while usage of > such constructs is not discouraged by the org manual Isn’t the point of discussing depreciation now that we’d change the manual etc. to discourage it? 😛 FWIW, I have an extra 2c to add: I think that there’s a danger in having $…$ /almost/ behave like LaTeX, as the closer it is the more surprising the edge cases are, and due to the nature of Org there will /always/ be a collection of edge cases with $…$. By comparison, \(…\) has much less ’surprising’ behaviour. >> If we do deprecate support for $…$, it might also be a good idea to >> see if we can add a utility function which would make it easier for >> people to migrate existing documents to the new/alternative syntax. For >> the same reason it is hard to reliably parse $…$ syntax, we probably >> can’t automate that transition, but we should be able to reduce the effort >> required to update existing documents. Tim, as mentioned before I’m strongly in favour of a ~half decade transition including utility functions to shift existing TeX constructs to LaTeX, and adding warnings, well before dropping support. All the best, Timothy
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
On 28/01/2022 02:15, Tim Cross wrote: If we do deprecate support for $...$, it might also be a good idea to see if we can add a utility function which would make it easier for people to migrate existing documents to the new/alternative syntax. For the same reason it is hard to reliably parse $...$ syntax, we probably can't automate that transition, but we should be able to reduce the effort required to update existing documents. There is no point to discuss deprecation of $...$ and $$...$$ while usage of such constructs is not discouraged by the org manual info "(org) LaTeX fragmets" https://orgmode.org/manual/LaTeX-fragments.html and by the compact guide https://orgmode.org/guide/Embedded-LaTeX.html Even though org syntax says that support of these constructs ought to be removed, users expect that they should work well.
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
Ihor Radchenko writes: > Rudolf Adamkovič writes: > >> Let $r_i$ denote the \(i\)-th rotation of $t$ with a suffix of $\ell|t|$ >> characters deleted, for […] >> >> Me, if I could, I would pay money for this feature, for it would allow >> me to use $$ consistently, focusing on mathematics instead of markup >> idiosyncrasies of "rotation $i$" versus "\(i\)-th rotation". > > Would it improve things for you if we change how \(...\) _looks_ in Org > buffers? > > The problem with parsing is more than just supporting $i$-th and > similar. For example, AMS style guide explicitly advises against using > $i$-th in favour of $i$th [1]: > > Do not hyphenate “th” expressions: xth, not x-th or xth . > > We can theoretically make a change to support "-", but then it will be > logical to support $i$th as well. (If we don't some users will still be > confused after trying to write $i$th and then not getting the expected > results). In this question, it would make sense to implement > all-or-everything approach. Otherwise, confusion (like raised in this > thread) will be inevitable. > > However, from point of view of Org mode parser, supporting $i$th is a > nightmare. Remember that Org mode is _not_ LaTeX and we have to support > a lot more frivolous syntax (even in LaTeX, runaway $ is often a source > of cryptic compilation errors). Currently, we _must_ rely on heuristics > to determine $$-style latex fragments. I do not know any way to support > $$ syntax without creating deviations from LaTeX. Extending the > heuristics will not resolve the underlying ambiguity of $$ syntax, just > hide it within even more obscure cases. > > Given the raised concerns, may we solve the issue with too verbose > \(...\) unambiguous syntax using the following approach: > 1. Fontify \(...\) replacing the brackets with a single character. For >example: > > \(...\) -> ⁅...⁆ > > 2. Provide convenient way to input \(\) brackets through >electric-pair-mode or trough org-cdlatex-mode. > > Best, > Ihor > > [1] https://www.ams.org/publications/authors/AMS-StyleGuide-online.pdf +1. Just my $0.02 worth - I think this is the right approach. Retaining support for $..$ doesn't seem feasible given all the complexities it brings with it. The main objections to the alternative appear to centre around readability and inconvenience of having to type additional characters or dealing with muscle memory use to $...$. These are essentially interface issues and I think we can largely address them using existing Emacs facilities. This will reduce the change impact to that sub-set of org users accustomed to $...$ while bringing the benefit of a cleaner and potentially more efficient parser to all org users. If we do deprecate support for $...$, it might also be a good idea to see if we can add a utility function which would make it easier for people to migrate existing documents to the new/alternative syntax. For the same reason it is hard to reliably parse $...$ syntax, we probably can't automate that transition, but we should be able to reduce the effort required to update existing documents.
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
Hi, Ihor Radchenko writes: We can theoretically make a change to support "-", but then it will be logical to support $i$th as well. (If we don't some users will still be confused after trying to write $i$th and then not getting the expected results). I disagree. 1. The $…$- pattern is also used for other common constructions, such as $n$-dimensional, $K$-lipschitz, etc. As for $n$−th vs $n$th, both are commonly used. In french, $n$− is the correct one. 2. It does not logically follow that we should support $i$th as well, since, as you point out, it'd be impossible. One argument for the patch is that is it very simple. 3. The $n$th construction failing is not as confusing. One understands quickly what the limitation is, and several workarounds are available, whereas there's no good reason for the $n$− limitation, and it's harder to think of a workaround. I should mention that the zero-width space character can be used to work around both limitations. Given the raised concerns, may we solve the issue with too verbose \(...\) unambiguous syntax using the following approach: 1. Fontify \(...\) replacing the brackets with a single character. For example: \(...\) -> ⁅...⁆ 2. Provide convenient way to input \(\) brackets through electric-pair-mode or trough org-cdlatex-mode. If the $…$ syntax were to be deprecated, this would be a nice addition, indeed. As a user, I find it quite satisfactory (with some added utility to easily switch between the \(…\) and \[…\] syntax). Some possible drawbacks : - are the display hacks scalable in a large document, with many LaTeX snippets ? - this feature may still be hard to discover, turn on and use by users more concerned with mathematical content and less familiar with emacs features such as fontification, automatic insertion of complex delimiters and whatnot. - hiding the syntax feels a bit weird, although it is already done with emphasis markers. With respect to the possible deprecation of the $…$ syntax, the drawbacks and complexity of this alternative should be weighted against those of the current $…$ syntax, but no one has really spelled out what the latter are. We're trading complexity here for complexity there, fixing the false positives (but how common are they ?) and the false negatives (which can be an annoyance while writing a snippet). Thank you for bringing this up, Regards, -- Sébastien Miquel
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
Rudolf wrote: > Further, \(\) brings 100% more characters than $$, resulting in more noise in > the sentence. Now where did I put my APL keyboard... Cheers Rasmus
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
Hi, Tom Gillespie writes: The change is local and minor. We can't know that. I meant that the change to the syntax would be minor and local, with respect to the linked syntax document. #+begin_src org I spent $20 on food and was paid$-10 dollars by friends so I am down $10. #+end_src [...] #+begin_src org Text a $A_BASH_VAR Text b some-$-lisp-var #+end_src The proposed change would break any file with a pattern like this. As you say, your first example is a typo. As for the second example, it's a very uncommon pattern, and if such variable names occur in a document, they should either be inside src blocks, or inside verbatim or code markers. In both case, there's no breakage with respect to parsing, export and tangling. The fontification may fail though. It wont fail if your src blocks are fontified natively, and the fontification of $…$ can be globally disabled. We have no way of seeing every org file that users have written so we don't know the extent of the impact, and thus have to assume that there would be some impact. Making such a change with an unknown blast radius in the midst of considering removing support for that syntax altogether is inviting disaster. We can make an educated guess. It is quite likely that this change will fix more documents than it breaks. Hardly a disaster. Regards, -- Sébastien Miquel
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
Rudolf Adamkovič writes: > Let $r_i$ denote the \(i\)-th rotation of $t$ with a suffix of $\ell|t|$ > characters deleted, for […] > > Me, if I could, I would pay money for this feature, for it would allow > me to use $$ consistently, focusing on mathematics instead of markup > idiosyncrasies of "rotation $i$" versus "\(i\)-th rotation". Would it improve things for you if we change how \(...\) _looks_ in Org buffers? The problem with parsing is more than just supporting $i$-th and similar. For example, AMS style guide explicitly advises against using $i$-th in favour of $i$th [1]: Do not hyphenate “th” expressions: xth, not x-th or xth . We can theoretically make a change to support "-", but then it will be logical to support $i$th as well. (If we don't some users will still be confused after trying to write $i$th and then not getting the expected results). In this question, it would make sense to implement all-or-everything approach. Otherwise, confusion (like raised in this thread) will be inevitable. However, from point of view of Org mode parser, supporting $i$th is a nightmare. Remember that Org mode is _not_ LaTeX and we have to support a lot more frivolous syntax (even in LaTeX, runaway $ is often a source of cryptic compilation errors). Currently, we _must_ rely on heuristics to determine $$-style latex fragments. I do not know any way to support $$ syntax without creating deviations from LaTeX. Extending the heuristics will not resolve the underlying ambiguity of $$ syntax, just hide it within even more obscure cases. Given the raised concerns, may we solve the issue with too verbose \(...\) unambiguous syntax using the following approach: 1. Fontify \(...\) replacing the brackets with a single character. For example: \(...\) -> ⁅...⁆ 2. Provide convenient way to input \(\) brackets through electric-pair-mode or trough org-cdlatex-mode. Best, Ihor [1] https://www.ams.org/publications/authors/AMS-StyleGuide-online.pdf
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
> The change is local and minor. We can't know that. Consider for example someone that has the following line somewhere in their files. #+begin_src org I spent $20 on food and was paid$-10 dollars by friends so I am down $10. #+end_src Yes =paid$-10= is probably a typo that should have a space in between, but it could still be in a file and cause an issue. The more likely case would be of someone that has $ in the name of a variable that also uses dashes. For example if I have a list of variable names such as #+begin_src org Text a $A_BASH_VAR Text b some-$-lisp-var #+end_src The proposed change would break any file with a pattern like this. We have no way of seeing every org file that users have written so we don't know the extent of the impact, and thus have to assume that there would be some impact. Making such a change with an unknown blast radius in the midst of considering removing support for that syntax altogether is inviting disaster. Best, Tom
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
Hi, Tom Gillespie writes: Unfortunately this falls into the realm of changes to syntax. The current behavior is not a bug and is working as specified because hyphen minus (U+002D) does not count as punctuation for the purposes of org syntax. We should specify which chars count as punctuation in the syntax doc. As noted by Eric \(\) has no such restrictions. Fromhttps://orgmode.org/worg/dev/org-syntax.html#Entities_and_LaTeX_Fragments POST is any punctuation (including parentheses and quotes) or space character, or the end of line. With this patch, I also propose to change the specification accordingly. The change is local and minor. Regards, -- Sébastien Miquel
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
Eric S Fraga writes: > But is not necessary (and further complicates the issue of support > $...$ in org as recently discussed on this list) as you can simply > type \(n\)-th? What you call "not necessary", I call a game changer. As of now, one has to mix and match $$ and \(\) based on the context (such as "-th"). It makes mathematical writing in both harder to /read/ and harder to /change/. Consider: Let $r_i$ denote the \(i\)-th rotation of $t$ with a suffix of $\ell|t|$ characters deleted, for […] Me, if I could, I would pay money for this feature, for it would allow me to use $$ consistently, focusing on mathematics instead of markup idiosyncrasies of "rotation $i$" versus "\(i\)-th rotation". Further, \(\) brings 100% more characters than $$, resulting in more noise in the sentence. Rudy -- "Thinking is a momentary dismissal of irrelevancies." -- Richard Buckminster Fuller, 1969 Rudolf Adamkovič [he/him] Studenohorská 25 84103 Bratislava Slovakia
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
> The attached patch adds support for $…$ latex fragments followed by a > dash, such as $n$-th. Unfortunately this falls into the realm of changes to syntax. The current behavior is not a bug and is working as specified because hyphen minus (U+002D) does not count as punctuation for the purposes of org syntax. We should specify which chars count as punctuation in the syntax doc. As noted by Eric \(\) has no such restrictions. >From https://orgmode.org/worg/dev/org-syntax.html#Entities_and_LaTeX_Fragments > POST is any punctuation (including parentheses and quotes) or space > character, or the end of line. Best, Tom
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
On a personal note, I would be very happy to see this patch merged. It has been annoying me for years, see for instance this thread: https://emacs.stackexchange.com/questions/37207/org-latex-fragment-succeeded-by-a-dash-is-not-showing Goran
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
Hi, Eric S Fraga writes: But is not necessary (and further complicates the issue of support $...$ in org as recently discussed on this list) as you can simply type \(n\)-th? What complication are you referring to, precisely ? The patch is fairly trivial, and a similar extension was already implemented by Nicolas in 2017[1]. Yes, the $…$ syntax is redundant, but I do not think it follows that this change is unnecessary. This patch should have no bearing on the possible deprecation of the $…$ syntax. [1]: https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=c0369a798470763f8f3c69cf2079c3a194635feb Regards, -- Sébastien Miquel
Re: [PATCH] Add support for $…$ latex fragments followed by a dash
On Monday, 24 Jan 2022 at 16:42, Sébastien Miquel wrote: > The attached patch adds support for $…$ latex fragments followed by a > dash, such as $n$-th. But is not necessary (and further complicates the issue of support $...$ in org as recently discussed on this list) as you can simply type \(n\)-th? -- : Eric S Fraga, with org release_9.5.2-312-g5d05f5 in Emacs 29.0.50