Re: [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements)

[Ihor Radchenko]

Nathaniel Nicandro  writes:

>> 1. Do not allow ANSI sequences to intersect markup boundaries of the
>>same AST depth:
>>*bold * plain text  should not trigger fontification
>>*bold  /italic/ * should trigger
>>plain text  *bold* plain text  also should
>
> Just to make sure I'm getting you right.  You're saying that
> fontification should trigger if the sequences live in the same
> org-element-context?

> What about cases like:
>
> *bold* plain text 
> plain text *bold * paragraph end
>
> In the first case, should only "bold" be fontified? Since the sequence
> lives in the bold context.

> In the second, should only "text"? Since the sequence lives at a higher
> depth (the paragraph context, compared to the bold context). Or should
> it be that the fontification should extend to the end of the paragraph
> because the sequence lives at a higher depth?

I completely missed the point that  codes are not 
pairs, but switches; this is completely different from Org syntax.

So, let me re-consider where  codes are likely to be used in
practice:

1. Inside shell code blocks (src-block element)
2. Inside results of evaluation, which are usually fixed-width element,
   but might also be example-block, export-block, drawer, table, or
   other element.
3. Inside shell inline code blocks (inline-src-block object)
4. Inside results of evaluation of an inline code block - usually
   code/verbatim markup.

I think that the most reasonable approach to fontify ANSI sequences will
be the following:

1. We will consider ANSI within (a) all greater elements and lesser
   elements that have RESULTS affiliated keyword (indicating that they
   are result of code block evaluation); (b) otherwise, just lesser
   elements, like paragraph, src block, example block, export block,
   etc., but _not_ tables (c) otherwise, within verbatim-like objects,
   like code, export-snippet, inline-src-block, table-cell, verbatim.

   The three groups above should be declared via variables, so that
   users can tweak them as necessary.

2. If ANSI sequence is encountered inside a verbatim-like object and we
   did not see any ANSI sequences within parent element or greater
   element, limit ANSI triggers to the current object.

   Example:

   #+RESULTS:
   Lorem upsum =valor=. Some more text.

   (only "valor" will be affected)

3. If the first ANSI sequence is encountered inside element and outside
   verbatim-like object, the rest of the element is affected, including
   all the objects.

   Example:

   #+RESULTS:
   Lorem upsum =valor=. Some more text.

   (the first ANSI affects everything, including verbatim; the second
   ANSI also affects everything)

4. If the first ANSI sequence is encountered inside greater element with
   RESULTS affiliated keyword, all the lesser elements inside will be
   affected.

   Example:

   #+RESULTS:
   :drawer:
   Lorem upsum =valor=. Some more text.

   Another paragraph inside drawer.
   :end:

   (everything down to :end: is affected)

   or

   #+RESULTS:
   - list
   - one
   - two
   - three

   (everything is affected down to the end of the list)

Does it make sense?

>>> +(cl-letf (((symbol-function #'delete-region)
>>> +   (lambda (beg end)
>>> + (add-text-properties beg end '(invisible t
>>
>> This is fragile and relies on internal implementation details of
>> ansi-color.el. Is there another way?
>
> Since the context in which the sequences live in need to be considered,
> it doesn't look like ansi-color-apply-on-region can be used any more
> since it isn't aware of Org objects.
>
> I've come up with a function that calculates the highlightable regions
> (considering contexts) and fontifies them, but it requires the use of
> private functions from ansi-color. Specifically
> ansi-color--face-vec-face, ansi-color--update-face-vec, and
> ansi-color--code-as-hex (used internally by ansi-color--face-vec-face).
> Does it make sense to copy over these functions into Org for the
> purposes of handling ANSI escapes? There would be some backward
> compatibility issues, e.g. ansi-color only started using faces as colors
> in Emacs 28.

If we really need to, we can propose an extension of
ansi-color-apply-on-region upstream for Emacs itself.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Formulas on table cells containing '$'

[Bruno Barbier]

Jeff Trull  writes:

> While investigating an error executing a table formula I discovered that
> cells containing '$' cause column references to be executed even when no
> attempt is made to evaluate cell contents as code. Here's a simple example:
>

Confirmed.


org tries first to resolve all references, recursively.


   '(length '(@1$3..@I$3))

   { resolving @1$3..@I$3 into ("$200.00" "$1.13" "$301.22") }
  
=  '(length '("$200.00" "$1.13" "$301.22"))

   { resolving $200 into ...
  
ERROR: '$200' is an invalid reference.



It's probably not by designed, but, it's definitely a limitation of
the current implementation.


As a workaround, you could hide your reference in elisp and resolve it
manually using `org-table-get-range'. That way you can add/remove "$" as
needed.


For example:

 | 3/1/2023  | Deposit| $200.00 |
 | 3/13/2023 | Interest   | $1.13   |
 | 4/1/2023  | Deposit| $301.22 |
 |---++-|
 |   | Number of Transactions | 3   |
 |   | Total  | $502.35 |
 #+TBLFM: @4$3='(length (org-table-get-range (concat "@" "1$" "3..@" "I$" 
"3")))
 #+TBLFM: @5$3='(format "$%.2f" (apply '+ (mapcar (lambda (x) 
(string-to-number (substring x 1))) (org-table-get-range (concat "@" "1$" 
"3..@" "I$" "3")




Bruno

Re: [POLL] Will it be ok to allow HABIT property inheritance? (optionally)

[Colin Baxter]

> Ihor Radchenko  writes:

> Colin Baxter  writes:
>> > No objections have been raised.
>> 
>> Well, you have not really given much time to respond.

> 2 weeks. I could have waited one month, but this is relatively
> safe breaking change.

>> Does this mean that all sub-heading with their TODOs will now
>> become habits? I would like much more information on the effects
>> of this proposal.

> Sorry if I was not clear in my original message.

> 1. STYLE will _not_ be inherited by default after the proposed
> change.  2. HABIT will be inherited if user customized
> `org-use-property-inheritance' to value of t (aka inherit all
> properties) 3. STYLE will be inherited if user explicitly added
> "STYLE" to `org-use-property-inheritance' when it is a list.

> Only (2) can be breaking.

Ok, that's good.

>> We have had `habits' for zillions of years which have worked
>> well. So why the change?

> To be able to dedicate a separate subtree or file to habits,
> automatically making all the sub-tasks habits without spamming
> property drawers setting :STYLE: habit.

Thank you very much. I understand.

Re: [POLL] Will it be ok to allow HABIT property inheritance? (optionally)

[Ihor Radchenko]

Colin Baxter  writes:

> > No objections have been raised.
>
> Well, you have not really given much time to respond.

2 weeks. I could have waited one month, but this is relatively safe
breaking change.

> Does this mean that all sub-heading with their TODOs will now become
> habits? I would like much more information on the effects of this
> proposal.

Sorry if I was not clear in my original message.

1. STYLE will _not_ be inherited by default after the proposed change.
2. HABIT will be inherited if user customized
   `org-use-property-inheritance' to value of t (aka inherit all properties)
3. STYLE will be inherited if user explicitly added "STYLE" to
   `org-use-property-inheritance' when it is a list.

Only (2) can be breaking.

> We have had `habits' for zillions of years which have worked well. So
> why the change?

To be able to dedicate a separate subtree or file to habits,
automatically making all the sub-tasks habits without spamming property
drawers setting :STYLE: habit.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [POLL] Will it be ok to allow HABIT property inheritance? (optionally)

[Colin Baxter]

> Ihor Radchenko  writes:

> Ihor Radchenko  writes:
>> I am generally in favour of allowing STYLE to be inherited, but
>> it is technically a breaking change, as I described.
>> 
>> Let's see if anyone voices against this change.

> No objections have been raised.

Well, you have not really given much time to respond.

Does this mean that all sub-heading with their TODOs will now become
habits? I would like much more information on the effects of this
proposal.

We have had `habits' for zillions of years which have worked well. So
why the change?

Re: [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements)

[Nathaniel Nicandro]

Ihor Radchenko  writes:

> Nathaniel Nicandro  writes:
>
>> The attached patch now uses `org-element-at-point' and
>> `org-element-context' to query for the bounds of elements.
>
> Thanks!
>
>> Note, I've also attached an updated example file which shows that the
>> escape sequences in inline source blocks are now handled similarly to
>> regular source blocks, i.e. they are not fontified.
>
> I do not think that a single exception - source blocks is good enough.
> When having something like
>   ANSI opening term is ==, and closing term is ==
> it will be not expected to get things fontified.
>
> A better approach will be:
> 1. Do not allow ANSI sequences to intersect markup boundaries of the
>same AST depth:
>*bold * plain text  should not trigger fontification
>*bold  /italic/ * should trigger
>plain text  *bold* plain text  also should

Just to make sure I'm getting you right.  You're saying that
fontification should trigger if the sequences live in the same
org-element-context?

What about cases like:

*bold* plain text 
plain text *bold * paragraph end

In the first case, should only "bold" be fontified? Since the sequence
lives in the bold context.

In the second, should only "text"? Since the sequence lives at a higher
depth (the paragraph context, compared to the bold context). Or should
it be that the fontification should extend to the end of the paragraph
because the sequence lives at a higher depth?

> 2. Disallow fontification is certain contexts - 'inline-src-block

What I will do then is not consider sequences in inline-src-block, code,
or verbatim contexts. Are there any other elements or objects that I
should not consider (other than the greater elements as you mention
below)?

For verbatim (and code) contexts, if there are regions like

 plain == text 

ANSIy will not get considered and the region between ANSIx and ANSIz
will get highlighted using ANSIx's settings.  So the verbatim object
gets highlighted as well.

For inline source blocks, I'll do what I did in the last patch and
decompose a paragraph into regions that exclude inline source blocks and
only consider those regions when processing the sequences. That way the
highlighting doesn't spill over into the inline source blocks (and not
interfere with the syntax highlighting of them).

>
> Further, your current code will do something weird when encountering
> greater element:
>
> :DRAWER:
> Paragraph 
>
> Another paragraph 
> :END:
>
> You should not consider greater elements when fontifying.
>

Thanks. In the case of greater elements, then, I will only consider
their contents.

For plain-lists and tables I will:
1. (for plain-lists) only consider the contents of the list items
2. (for tables) only consider the table-cells of each table-row

>> +(cl-letf (((symbol-function #'delete-region)
>> +   (lambda (beg end)
>> + (add-text-properties beg end '(invisible t
>
> This is fragile and relies on internal implementation details of
> ansi-color.el. Is there another way?

Since the context in which the sequences live in need to be considered,
it doesn't look like ansi-color-apply-on-region can be used any more
since it isn't aware of Org objects.

I've come up with a function that calculates the highlightable regions
(considering contexts) and fontifies them, but it requires the use of
private functions from ansi-color. Specifically
ansi-color--face-vec-face, ansi-color--update-face-vec, and
ansi-color--code-as-hex (used internally by ansi-color--face-vec-face).
Does it make sense to copy over these functions into Org for the
purposes of handling ANSI escapes? There would be some backward
compatibility issues, e.g. ansi-color only started using faces as colors
in Emacs 28.

-- 
Nathaniel

Re: [POLL] Will it be ok to allow HABIT property inheritance? (optionally) (was: [BUG] org-habit does not respect STYLE property inheritance)

[Ihor Radchenko]

Ihor Radchenko  writes:

> I am generally in favour of allowing STYLE to be inherited, but it is
> technically a breaking change, as I described.
>
> Let's see if anyone voices against this change.

No objections have been raised.

Dmitry, could you please create a proper patch, adding etc/ORG-NEWS
entry that describes the change (under " ... breaking changes" section),
and a proper commit message? See
https://orgmode.org/worg/org-contribute.html#first-patch

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: ob-clojure: results quoted nil

[Ihor Radchenko]

Daniel Kraus  writes:

>> Even if it is the case, it is still worth checking :results table
>> parsing. nils might be slurped when converting output to tables.
>
> Right.
> I just double checked the results for value and table results with
> different Clojure backends and it worked in all cases (that I tested).

Thanks! So, we see no problems here.

Canceled.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [ANN] lisp/ob-tangle-sync.el

[Ihor Radchenko]

Mehmet Tekman  writes:

>> See the above 3 examples: (1) "foo.txt" shadowed by "no"; (2) "no"
>> shadowed by "foo.txt"; (3) "no" shadowed by "foo.txt" then shadowed by
>> "yes".
>>
>> You can also consider "foo.txt" shadowed by "bar.txt" and other
>> variations.
>
> I've created a small patch that contains a single ert test function that
> checks an example org file I made (based on the above as well as my own
> toy file) to try to validate the `org-babel-merge-params' rewrite.
>
> I've attached it below in case you want to test it...

Is the patch testing your code?

(Note on the ert tests: we generally prefer `org-test-with-temp-text' -
the old approach with test IDs is not very readable because one needs to
search those IDs manually when debugging test failures.) 

> ...but there were a few cases where I wasn't entirely sure what the
> result of the merge was supposed to be:
>
> For example, a document with:
>
>> #+TITLE: Header tests
>> #+PROPERTY: header-args :tangle /tmp/default_tangle.txt
>> 
>> * Inherit tangle header from document
>> 
>> #+begin_src conf
>>   (:tangle . /tmp/default_tangle.txt)
>> #+end_src
>
> I would expect the output of:
>
> (assoc :tangle (nth 2 (org-babel-get-src-block-info)))
>
> within that block to evaluate to the contents written in that
> block. Instead it evaluates to `(:tangle . no)' when run in a vanilla
> Emacs. Is this expected?

You forgot C-c C-c on the PROPERTY line, or re-opening the file.
#+PROPERTY lines are not automatically parsed when you add them. They
are only parsed upon explicit request (C-c C-c) or when opening the
file.

> Another example:
>
>> * Header args overwritten by local header
>> :PROPERTIES:
>> :header-args: :tangle "foo.txt"
>> :END:
>> #+begin_src :tangle yes
>>(:tangle . foo.txt)
>> #+end_src
>>
>> ** Inherited header
>> 
>> #+begin_src :tangle "file with spaces.txt"
>>   (:tangle . "file with spaces.txt")
>> #+end_src
>
> The first block correctly gives "foo.txt" under vanilla Emacs, but the
> second block also gives "foo.txt". Is this expected behaviour?

You forgot to specify the source block languages. Org sees your example
as two src blocks of language ":tangle" with no proper header arguments.
(See M-x org-lint)

If you add a language, like 
 #+begin_src conf :tangle yes
(:tangle . foo.txt)
 #+end_src

The expected result is (:tangle . "yes") for the first block and
(:tangle . "file with spaces.txt") for the second block.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Different faces for breadcrumbs and main entry in agenda

[Vladimir Nikishkin]

Hello, everyone.

Can someone suggest how I could implement different colours for
breadcrumbs and for the main entry name in the agenda view?

I really need this, in my case it would greatly improve readability.

-- 
Your sincerely,
Vladimir Nikishkin (MiEr, lockywolf)
(Laptop)