Sanitize bidi control chars
* 2020-08-10 19:45:11+03, Teemu Likonen wrote: > If we wanted to clean message headers from possible unpaired overrides > we should clean all these: > > U+202A LEFT-TO-RIGHT EMBEDDING (push) > U+202B RIGHT-TO-LEFT EMBEDDING (push) > U+202C POP DIRECTIONAL FORMATTING (pop) > U+202D LEFT-TO-RIGHT OVERRIDE (push) > U+202E RIGHT-TO-LEFT OVERRIDE (push) > > Or we could even try to be clever and count those characters and then > insert or remove some of them so that there are as many "push" > characters as "pop" characters. Below is an example Emacs Lisp function to balance those "push" and "pop" bidi control chars. This kind of code could be used to sanitize message headers or any arbitrary text coming from user. I'm not even sure if such thing should be done in Emacs or in lower level Notmuch code. Anyway, I tried to add it to notmuch-sanitize function. Now Tomi's message didn't switch direction of other text anymore (in notmuch-search-mode buffer). (defun notmuch-balance-bidi-ctrl-chars (string) (let ((new nil) (stack-count 0)) (cl-flet ((push-char-p (c) ;; U+202A LEFT-TO-RIGHT EMBEDDING ;; U+202B RIGHT-TO-LEFT EMBEDDING ;; U+202D LEFT-TO-RIGHT OVERRIDE ;; U+202E RIGHT-TO-LEFT OVERRIDE (cl-find c '(?\x202a ?\x202b ?\x202d ?\x202e))) (pop-char-p (c) ;; U+202C POP DIRECTIONAL FORMATTING (eql c ?\x202c))) (cl-loop for char across string do (cond ((push-char-p char) (cl-incf stack-count) (push char new)) ((and (pop-char-p char) (cl-plusp stack-count)) (cl-decf stack-count) (push char new)) ((and (pop-char-p char) (not (cl-plusp stack-count))) ;; The stack is empty. Ignore this pop char. ) (t (push char new) ;; Add missing pops. (cl-loop repeat stack-count do (push ?\x202c new)) (seq-into (nreverse new) 'string))) -- /// Teemu Likonen - .-.. http://www.iki.fi/tlikonen/ // OpenPGP: 4E1055DC84E9DFF613D78557719D69D324539450 signature.asc Description: PGP signature ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH v5] Emacs: Ensure left-to-right display for message headers
* 2020-08-09 23:12:28+03, utf wrote: > How about this =D > From: contains U+202E (LEFT-TO-RIGHT OVERRIDE) (in > =?utf-8?Q?T=E2=80=AEomi?=) Indeed message's header fields can contain such override characters. The override mode should be terminated with U+202C POP DIRECTIONAL FORMATTING within the same header field. That POP character pops the override mode from the direction mode stack and returns to the previous mode. Those characters can mess any text badly when not used in controlled pairs of "push" and "pop". I'll write "abc abc abc" series but the middle "abc" have RIGHT-TO-LEFT OVERRIDE before "a" and POP DIRECTIONAL FORMATTING after "c". In Emacs try using C-f and C-b commands to move the cursor above the text: abc abc abc If we wanted to clean message headers from possible unpaired overrides we should clean all these: U+202A LEFT-TO-RIGHT EMBEDDING (push) U+202B RIGHT-TO-LEFT EMBEDDING (push) U+202C POP DIRECTIONAL FORMATTING (pop) U+202D LEFT-TO-RIGHT OVERRIDE (push) U+202E RIGHT-TO-LEFT OVERRIDE (push) Or we could even try to be clever and count those characters and then insert or remove some of them so that there are as many "push" characters as "pop" characters. -- /// Teemu Likonen - .-.. http://www.iki.fi/tlikonen/ // OpenPGP: 4E1055DC84E9DFF613D78557719D69D324539450 signature.asc Description: PGP signature ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH v2] Emacs: Indent first header line only when indentation is turned on
On Monday, 2020-08-10 at 17:37:57 +03, Teemu Likonen wrote:
> Previously in message-show mode message's first header line (From
> header) was always indented, even if user had turned thread
> indentation off with "<" (notmuch-show-toggle-thread-indentation)
> command.
>
> This change modifies notmuch-show-insert-headerline function so that
> it doesn't indent the first header line if notmuch-show-indent-content
> variable is nil.
Reviewed-by: David Edmondson
> ---
> emacs/notmuch-show.el | 5 -
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
>
> * 2020-08-10 11:19:10+01, David Edmondson wrote:
>>> +(insert (notmuch-show-spaces-n
>>> +(if notmuch-show-indent-content
>>> +(* notmuch-show-indent-messages-width depth)
>>> + 0))
>>
>> Couldn't you also elide the call to `notmuch-show-spaces-n'?
>
> Indeed.
>
>
>
> diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
> index 0eb27e33..444b2a45 100644
> --- a/emacs/notmuch-show.el
> +++ b/emacs/notmuch-show.el
> @@ -474,7 +474,10 @@ message at DEPTH in the current thread."
>;; invisible U+200E LEFT-TO-RIGHT MARK character which forces
>;; the header paragraph as left-to-right text.
>(insert (propertize (string ?\x200e) 'invisible t)))
> -(insert (notmuch-show-spaces-n (* notmuch-show-indent-messages-width
> depth))
> +(insert (if notmuch-show-indent-content
> + (notmuch-show-spaces-n (* notmuch-show-indent-messages-width
> + depth))
> + "")
> from
> " ("
> date
> --
> 2.20.1
dme.
--
I used to get mad at my school, the teachers who taught me weren't cool.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
[PATCH v2] Emacs: Indent first header line only when indentation is turned on
Previously in message-show mode message's first header line (From
header) was always indented, even if user had turned thread
indentation off with "<" (notmuch-show-toggle-thread-indentation)
command.
This change modifies notmuch-show-insert-headerline function so that
it doesn't indent the first header line if notmuch-show-indent-content
variable is nil.
---
emacs/notmuch-show.el | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
* 2020-08-10 11:19:10+01, David Edmondson wrote:
>> +(insert (notmuch-show-spaces-n
>> + (if notmuch-show-indent-content
>> + (* notmuch-show-indent-messages-width depth)
>> + 0))
>
> Couldn't you also elide the call to `notmuch-show-spaces-n'?
Indeed.
diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 0eb27e33..444b2a45 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -474,7 +474,10 @@ message at DEPTH in the current thread."
;; invisible U+200E LEFT-TO-RIGHT MARK character which forces
;; the header paragraph as left-to-right text.
(insert (propertize (string ?\x200e) 'invisible t)))
-(insert (notmuch-show-spaces-n (* notmuch-show-indent-messages-width
depth))
+(insert (if notmuch-show-indent-content
+ (notmuch-show-spaces-n (* notmuch-show-indent-messages-width
+ depth))
+ "")
from
" ("
date
--
2.20.1
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] Emacs: Indent first header line only when indentation is turned on
On Sunday, 2020-08-09 at 22:01:44 +03, Teemu Likonen wrote:
> Previously in message-show mode message's first header line (From
> header) was always indented, even if user had turned thread
> indentation off with "<" (notmuch-show-toggle-thread-indentation)
> command.
>
> This change modifies notmuch-show-insert-headerline function so that
> it doesn't indent the first header line if notmuch-show-indent-content
> variable is nil.
> ---
> emacs/notmuch-show.el | 5 -
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
> index 0eb27e33..2310017a 100644
> --- a/emacs/notmuch-show.el
> +++ b/emacs/notmuch-show.el
> @@ -474,7 +474,10 @@ message at DEPTH in the current thread."
>;; invisible U+200E LEFT-TO-RIGHT MARK character which forces
>;; the header paragraph as left-to-right text.
>(insert (propertize (string ?\x200e) 'invisible t)))
> -(insert (notmuch-show-spaces-n (* notmuch-show-indent-messages-width
> depth))
> +(insert (notmuch-show-spaces-n
> + (if notmuch-show-indent-content
> + (* notmuch-show-indent-messages-width depth)
> +0))
Couldn't you also elide the call to `notmuch-show-spaces-n'?
> from
> " ("
> date
> --
> 2.20.1
> ___
> notmuch mailing list -- notmuch@notmuchmail.org
> To unsubscribe send an email to notmuch-le...@notmuchmail.org
dme.
--
I walk like a building, I never get wet.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH v5] Emacs: Ensure left-to-right display for message headers
On Sunday, 2020-08-09 at 23:12:28 +03, =?utf-8?Q?T=E2=80=AEomi?= Ollila wrote:
> How about this =D
Your point being that the code sanitising the displayed header could do
better?
> (Sorry how the headers might look...)
>
> From: contains U+202E (LEFT-TO-RIGHT OVERRIDE) (in =?utf-8?Q?T=E2=80=AEomi?=)
>
> https://www.fileformat.info/info/unicode/char/202e/index.htm
>
> Tomi
>
> ---
>
> (top-posting on purpose, all rest is for reference only)
>
> On Fri, Aug 07 2020, Teemu Likonen wrote:
>
>> In notmuch-show buffer insert invisible U+200E LEFT-TO-RIGHT MARK
>> character at the beginning of message header paragraph if the From
>> header contains a right-to-left character. This ensures that the
>> header paragraph is always rendered in left-to-right mode.
>>
>> See Emacs Lisp reference manual section "(elisp) Bidirectional
>> Display" for more info.
>> ---
>> emacs/notmuch-show.el | 12 +---
>> 1 file changed, 9 insertions(+), 3 deletions(-)
>>
>>
>> As the commit description says this version inserts U+200E
>> LEFT-TO-RIGHT MARK only if the first header line (From header)
>> contains a right-to-left character.
>>
>> This version is probably friendlier to the current test files which
>> don't expect to see U+200E LEFT-TO-RIGHT MARK in the output.
>>
>>
>> diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
>> index c9170466..0eb27e33 100644
>> --- a/emacs/notmuch-show.el
>> +++ b/emacs/notmuch-show.el
>> @@ -466,10 +466,16 @@ unchanged ADDRESS if parsing fails."
>> (defun notmuch-show-insert-headerline (headers date tags depth)
>>"Insert a notmuch style headerline based on HEADERS for a
>> message at DEPTH in the current thread."
>> - (let ((start (point)))
>> + (let ((start (point))
>> +(from (notmuch-sanitize
>> + (notmuch-show-clean-address (plist-get headers :From)
>> +(when (string-match "\\cR" from)
>> + ;; If the From header has a right-to-left character add
>> + ;; invisible U+200E LEFT-TO-RIGHT MARK character which forces
>> + ;; the header paragraph as left-to-right text.
>> + (insert (propertize (string ?\x200e) 'invisible t)))
>> (insert (notmuch-show-spaces-n (* notmuch-show-indent-messages-width
>> depth))
>> -(notmuch-sanitize
>> - (notmuch-show-clean-address (plist-get headers :From)))
>> +from
>> " ("
>> date
>> ") ("
>> --
>> 2.20.1
> ___
> notmuch mailing list -- notmuch@notmuchmail.org
> To unsubscribe send an email to notmuch-le...@notmuchmail.org
dme.
--
Our President's crazy, did you hear what he said?
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
