Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Ihor Radchenko]

Bruno Barbier  writes:

> ... I'm now using using a fully working example.

Thanks!
I will first reply to you email inline and then common further on your
changes from the branch.

>>> ;;(org-pending-send-update my-rlock  (list :progress "Not ready 
>>> yet."))
>>> ;;(org-pending-send-update my-rlock  (list :progress "Coming 
>>> soon."))
>>
>> Should the progress message always be a string?
>
> No.  It may currently be any data.  org-pending will format it as a
> string that fits on one line.

Please say this in the docstring of `org-pending-send-update'.

>>> ;;(org-pending-send-update my-rlock (list :success 1))
>>
>> What will org-pending do with :success 1? Will it replace region with
>> "1" or will it do something else?
>
> That's the job on ON-OUTCOME to convert/format/append/prepend/replace
> some content if needed, on :success and/or on :failure.

Fair. Although, it feels like a common use case to replace the region
with :success value. Maybe the library should provide some ready-to-use
functions that can be used as the value of :on-outcome.

>> It would be nice to have an example that will also send a signal to
>> process, as it is probably the most commonly used way to utilize
>> org-pending.
>
> For my many use cases, that would always be a mistake to kill the
> process: an OS process is always in charge of many locks.
>
> More importantly, to find a self-contained working readable example
> might be a challenge.
>
> We could add a function 'org-pending-shell-command-on-region' in
> org-pending, that could be used as an implementation example, like
> `org-pending-user-edit', `org-babel-execute-src-block', etc.

Yes, having pre-cooked wrappers for `org-pending' or pre-defined values
for :on-outcome/:befire-kill-function/:user-cancel-function/etc would be useful.

;; ;; We lock the 'region', defining how to update it when the
;; ;; outcome is available.
;; (setq my-lock (org-pending
;;region
;;:on-outcome
;;(lambda (_rl outcome)
;;  (pcase outcome
;;(`(:success ,value)
;; ;; On :success, we replace the region with the
;; ;; value.
;; (let ((tmp-end (cdr region)))
;;   (goto-char tmp-end)
;;   (insert (format "%s\n" value))
;;   (setcdr region (point))
;;   (delete-region (car region) tmp-end)))
;; ...

This example has a major problem if user edits the buffer text _before_
locked region before the outcome is available.
(car region) and (cdr region) will no longer be accurate, and your code
will replace text in places you do not expect.

I believe that it will be better to query region-lock object about the
region location where we need to replace text:

(setq region (org-pending-reglock-region rl))

Same for reglock buffer in other examples.

Then, we will keep the possibility open for org-pending to handle cases
like killing/yanking text containing reglocks (org-refile) - org-pending
may in future keep track of them.

;; (setf (org-pending-reglock-user-cancel-function my-lock)
;;   (let ((this-timer my-timer))
;; (lambda (_rl)
;;   (cancel-timer this-timer)
;; ...

;; (setf (org-pending-reglock-before-kill-function my-lock)
;;   (let ((this-timer my-timer))
;; (lambda (_rl)
;;   (message "Killing %s" this-timer)
;;   (cancel-timer this-timer

What is the difference between "canceling" and "killing" the reglock?
Do they need to be separate?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Bruno Barbier]

Ihor Radchenko  writes:

> Bruno Barbier  writes:
>
> I am attaching some minor edits I'd like to propose on top of your
> latest branch.

I've applied your patch. Thanks.

> Also, some more questions.
>
>> ;;(setq my-rlock
>> ;;  (org-pending (cons (point) (mark))
>> ;;   (lambda (outcome)
>> ;; (pcase outcome
>> ;;   (`(:success ,result) (goto-char END) 
>> (insert result))
>> ;;   (`(:failure ,err) (message "Failed: %s" 
>> err))
>
> 1. It is more natural in Elisp to pass regions as two parameters: BEG
>and END, not a cons cell.

I rewrote the example so that it's complete and can be uncommented and
evaluated.

With the new example, it's less obvious which is more natural.



> 2. Note that (point) may be _after_ (mark).  AFAIU, you code assumes
>that point is always before the mark.  You may want to address this.

Right. I'm not using those in the new example: problem solved ;)



> 3. ON-OUTCOME is optional. What happens if none is provided?

No function is called. And thus, no value is returned, so no mark.
It's like calling a function that does nothing and return nil.



> 4. In the `org-pending' docstring you say "ON-OUTCOME is non-nil, call
>it with the reglock and the outcome", but the example shows a lambda
>accepting a single argument. Something is off. I'm afraid that this
>example will not work if copy-pasted.
>

You're right. Sorry. I'm now using using a fully working example.



>> ;;(org-pending-send-update my-rlock  (list :progress "Not ready 
>> yet."))
>> ;;(org-pending-send-update my-rlock  (list :progress "Coming soon."))
>
> Should the progress message always be a string?

No.  It may currently be any data.  org-pending will format it as a
string that fits on one line.


>> ;;(org-pending-send-update my-rlock (list :success 1))
>
> What will org-pending do with :success 1? Will it replace region with
> "1" or will it do something else?

That's the job on ON-OUTCOME to convert/format/append/prepend/replace
some content if needed, on :success and/or on :failure.


>> ;;(org-pending-send-update my-rlock (list :failure "Some error!"))
>
> I am slightly confused by this calling convention. Why not simply
>
> (org-pending-send-update my-rlock :failure "Some error!")

Because one message update is one value, either:
 (list :success value)
or
 (list :failure err)
or
 (list :progress data)

On :success/:failure, this message is also a valid outcome, and it's
the same value that ON-OUTCOME will receive, as one single entity.

When listing artificial calls like this, it does look suboptimal; but,
in general, it's simpler to have a real value we can store, pass
around and return.

>
>> ;;(setf (org-pending-reglock-insert-details-function my-reglock)
>> ;;  (lambda (rl _start _end)
>> ;;(insert (format "%s" (org-pending-reglock-property rl 
>> :my-prop)
>
> Are there any standard properties? It would be nice to list them in a
> table as well.

There are no standard properties.  All required properties have been
hard-coded in the cl-defstruct.

> Also, you can show an example of _setting_ :my-prop property.

I removed the use of `org-pending-reglock-property' from the example; I
think it was a mistake to introduce that advanced feature here.  The
documentation of the function `org-pending-reglock-property' explains
how to set a property.


>> ;; If the user kills a buffer, or, kills Emacs, some locks may have to
>> ;; be killed too.  The library will ask the user to confirm if an
>> ;; operation requires to kill some locks.  See the field
>> ;; `before-kill-function' of REGLOCK object, if you need to do
>> ;; something before a lock is really killed.  For example, if you like
>> ;; to kill a MY-BUFFER before MY-LOCK is killed, you can do:
>> ;;
>> ;;(setf (org-pending-reglock-before-kill-function my-reglock)
>> ;;  (lambda (_rl) (kill-buffer my-buffer)))
>
> It would be nice to have an example that will also send a signal to
> process, as it is probably the most commonly used way to utilize
> org-pending.

For my many use cases, that would always be a mistake to kill the
process: an OS process is always in charge of many locks.

More importantly, to find a self-contained working readable example
might be a challenge.

We could add a function 'org-pending-shell-command-on-region' in
org-pending, that could be used as an implementation example, like
`org-pending-user-edit', `org-babel-execute-src-block', etc.


> From `org-pending' docstring:
>
>> If ON-OUTCOME returns
>> a region (a pair (start position . end position)), use it to report the
>> success/failure using visual hints on that region.  If ON-OUTCOME
>> returns nothing, don't display outcome marks.
>
> What if ON-OUTCOME returns something that is not a cons cell and not nil?
>

Then org-pending would have 

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Ihor Radchenko]

Bruno Barbier  writes:

> I've pushed the update to my public branch.

Thanks!

I am attaching some minor edits I'd like to propose on top of your
latest branch.

Also, some more questions.

> ;;(setq my-rlock
> ;;  (org-pending (cons (point) (mark))
> ;;   (lambda (outcome)
> ;; (pcase outcome
> ;;   (`(:success ,result) (goto-char END) (insert 
> result))
> ;;   (`(:failure ,err) (message "Failed: %s" 
> err))

1. It is more natural in Elisp to pass regions as two parameters: BEG
   and END, not a cons cell.

2. Note that (point) may be _after_ (mark).  AFAIU, you code assumes
   that point is always before the mark.  You may want to address this.

3. ON-OUTCOME is optional. What happens if none is provided?

4. In the `org-pending' docstring you say "ON-OUTCOME is non-nil, call
   it with the reglock and the outcome", but the example shows a lambda
   accepting a single argument. Something is off. I'm afraid that this
   example will not work if copy-pasted.

> ;;(org-pending-send-update my-rlock  (list :progress "Not ready 
> yet."))
> ;;(org-pending-send-update my-rlock  (list :progress "Coming soon."))

Should the progress message always be a string?

> ;;(org-pending-send-update my-rlock (list :success 1))

What will org-pending do with :success 1? Will it replace region with
"1" or will it do something else?

> ;;(org-pending-send-update my-rlock (list :failure "Some error!"))

I am slightly confused by this calling convention. Why not simply

(org-pending-send-update my-rlock :failure "Some error!")

> ;;(setf (org-pending-reglock-insert-details-function my-reglock)
> ;;  (lambda (rl _start _end)
> ;;(insert (format "%s" (org-pending-reglock-property rl 
> :my-prop)

Are there any standard properties? It would be nice to list them in a
table as well.

Also, you can show an example of _setting_ :my-prop property.

> ;; If the user kills a buffer, or, kills Emacs, some locks may have to
> ;; be killed too.  The library will ask the user to confirm if an
> ;; operation requires to kill some locks.  See the field
> ;; `before-kill-function' of REGLOCK object, if you need to do
> ;; something before a lock is really killed.  For example, if you like
> ;; to kill a MY-BUFFER before MY-LOCK is killed, you can do:
> ;;
> ;;(setf (org-pending-reglock-before-kill-function my-reglock)
> ;;  (lambda (_rl) (kill-buffer my-buffer)))

It would be nice to have an example that will also send a signal to
process, as it is probably the most commonly used way to utilize
org-pending.


>From `org-pending' docstring:

> If ON-OUTCOME returns
> a region (a pair (start position . end position)), use it to report the
> success/failure using visual hints on that region.  If ON-OUTCOME
> returns nothing, don't display outcome marks.

What if ON-OUTCOME returns something that is not a cons cell and not nil?

diff --git a/lisp/org-pending.el b/lisp/org-pending.el
index 2530a95b3..973b5e17b 100644
--- a/lisp/org-pending.el
+++ b/lisp/org-pending.el
@@ -26,11 +26,18 @@ ;;; Commentary:
  Overview
 ;;
 ;;
-;; This library contains an API to lock a region while it is "being
+;; This library provides an API to lock a region while it is "being
 ;; updated"; the content of the region is "pending" and cannot be
 ;; modified.  It will be updated, later, when the new content is
 ;; available.
 ;;
+;; While region is "pending", the library will mark it for the user,
+;; displaying the current update progress.
+;;
+;; The update may yield success or failure.  On success, the region
+;; content will be updated, and the update summary will be indicated.
+;; On failure, the error log will be displayed.
+;;
 ;; Locking regions is useful when the update is computed
 ;; asynchronously and/or depends on external events.
 ;;
@@ -38,7 +45,7 @@  Overview
  How to use locks in your library
 ;;
 
-;; To lock a region, you need to do something like this:
+;; To lock a region, you need to:
 ;;
 ;;1. Call the function `org-pending' with the region to lock; use
 ;;   the ON-OUTCOME argument to tell Emacs how to update the
@@ -47,18 +54,19 @@  How to use locks in your library
 ;;
 ;;2. Start "something" that computes the new content.  That
 ;;   "something" may be a thread, a timer, a notification, a
-;;   process, etc.  That "something" must eventually send a
-;;   :success or :failure message (using
-;;   `org-pending-send-update'): Emacs will update the pending
-;;   region (using your ON-OUTCOME) and unlock it; at this point
-;;   the lock is "dead" (see `org-pending-reglock-live-p').
+;;   process, etc.  That "something" might optionally report
+;;   :progress, and must eventually send a :success or :failure
+;;   message (using `org-pending-send-update'): org-pending will
+;;   

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Bruno Barbier]

Hi Ihor,

Ihor Radchenko  writes:

> Bruno Barbier  writes:
>
>> I've pushed the modification to my branch.
>
> Thanks! Let's work further on the top comment.
>
>> ;; To lock a region, you need to do something like this:
>
> I think "something like this" can be just dropped.

Sorry, I'm failing to find a simpler accurate sentence.


>> ;;1. Call the function `org-pending' with the region to lock; use
>> ;;   the ON-OUTCOME argument to tell Emacs how to update the
>> ;;   region.  Keep the returned REGLOCK (you'll need it to send
>> ;;   updates).
>
> It would be nice to provide examples of `org-pending' call right in the
> top comment. Ideally, the example should also show how to modify REGLOCK
> fields to customize its behaviour.

I added an example below the 2 steps.

>> ;;2. Start "something" that computes the new content.  That
>> ;;   "something" may be a thread, a timer, a notification, a
>> ;;   process, etc.  That "something" must eventually send a
>> ;;   :success or :failure message (using
>> ;;   `org-pending-send-update'): Emacs will update the pending
>> ;;   region (using your ON-OUTCOME) and unlock it; at this point
>> ;;   the lock is "dead" (not live-p).
>
> Please also add information about sending updates to the REGLOCK, with
> examples. Otherwise, "receiving progress" in the next section is
> surprising.

Done in the example too.

>> ... (not live-p)
>
> Please name `org-pending-reglock-live-p' - it is more clear than forcing
> the readers search it themselves.

Done.

>
>>  Interface provided to the Emacs user
>> ;;
>> ;; The library makes locks visible to the user using text properties
>> ;; and/or overlays.  It diplays and updates the status while the
>> ;; region is locked: the initial status is "scheduled", then, when
>
> It would be nice to name `org-pending-reglock-status' here and use the
> actual status values - :scheduled, :pending, :success, :failure.
> Ideally, in table/list explaining what happens with buffer text,
> overlays, and user interaction, when REGLOCK has each of the listed
> status values.

I added a table describing the possible status values, the valid
updates, etc in the section above.


>
>> ;; receiving progress it becomes "pending" (with progress information
>> ;; if any).  Emacs allows to diplay a description of the lock.  From
>> ;; that description, the user may request to cancel that lock; see the
>> ;; field `user-cancel-function' of the REGLOCK object if you need to
>> ;; customize what to do on cancel.
>
> It is not very clear how user can interact with "description".
> Is it a  tooltip? A window? Something else? Please give a bit more details.

I added that it's like the function `describe-package'.

> Also, when "cancel" is requested, it is a good idea to state that
> `user-cancel-function' is called and describe what it does by default.

The function `user-cancel-function' is not always called; Emacs will
only call it if the lock is still live (see `org-pending-cancel').

I added a short sentence describing what is done by default (the
documentation of the field `user-cancel-function' of the reglock
object already fully document `user-cancel-function').


>> ;; When receiving the outcome (success or failure), after unlocking
>> ;; the region, the library may leave information about the outcome
>
> "may"?? Or does it always leave the information (by default)?

To mark the outcome, org-pending needs to know where it is.  If the
ON-OUTCOME function (see `org-pending` documentation) returns the
outcome region, org-pending will add a mark, else not.  By default,
ON-OUTCOME is nil, meaning no outcome region and no outcome marks.  I
added a sentence about that.


>> ;; (using text properties/overlays).  If that outcome information is
>> ;; (still) displayed, Emacs allows to display a description of that
>> ;; lock.  From that description, the user may decide to "forget" that
>> ;; lock; "forgetting the lock" removes the outcome visual marks, and,
>
> Is "forgetting" customizeable like `user-cancel-function'?

No.

>> ;; it allows Emacs to discard any information related to this lock.
>
> What does it mean?

Emacs is free to delete any data related to this lock.  This lock will
not be available anywhere anymore for the user. Concretely, this lock
is remove from the list of known locks, and, eventually, Emacs should
hopefully garbage collect the related data.

>
>> ;; The description of a lock (live or dead) provides information like
>> ;; the schedule time, the duration, the outcome time, the result (in
>> ;; case of success), the error (in case of failure), etc.  Customize
>> ;; the field `insert-details-function' of REGLOCK object to add your
>> ;; own information.
>
> Please show an example how to do it here.

Done.

>
>> ;; If the user kills a buffer, or, kills Emacs, some locks may have to
>> ;; be killed too be killed too.  The library will ask the user to
>  ^^^
> typo

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Ihor Radchenko]

Bruno Barbier  writes:

> I've pushed the modification to my branch.

Thanks! Let's work further on the top comment.

> ;; To lock a region, you need to do something like this:

I think "something like this" can be just dropped.

> ;;1. Call the function `org-pending' with the region to lock; use
> ;;   the ON-OUTCOME argument to tell Emacs how to update the
> ;;   region.  Keep the returned REGLOCK (you'll need it to send
> ;;   updates).

It would be nice to provide examples of `org-pending' call right in the
top comment. Ideally, the example should also show how to modify REGLOCK
fields to customize its behaviour.

> ;;2. Start "something" that computes the new content.  That
> ;;   "something" may be a thread, a timer, a notification, a
> ;;   process, etc.  That "something" must eventually send a
> ;;   :success or :failure message (using
> ;;   `org-pending-send-update'): Emacs will update the pending
> ;;   region (using your ON-OUTCOME) and unlock it; at this point
> ;;   the lock is "dead" (not live-p).

Please also add information about sending updates to the REGLOCK, with
examples. Otherwise, "receiving progress" in the next section is
surprising.

> ... (not live-p)

Please name `org-pending-reglock-live-p' - it is more clear than forcing
the readers search it themselves.

>  Interface provided to the Emacs user
> ;;
> ;; The library makes locks visible to the user using text properties
> ;; and/or overlays.  It diplays and updates the status while the
> ;; region is locked: the initial status is "scheduled", then, when

It would be nice to name `org-pending-reglock-status' here and use the
actual status values - :scheduled, :pending, :success, :failure.
Ideally, in table/list explaining what happens with buffer text,
overlays, and user interaction, when REGLOCK has each of the listed
status values.

> ;; receiving progress it becomes "pending" (with progress information
> ;; if any).  Emacs allows to diplay a description of the lock.  From
> ;; that description, the user may request to cancel that lock; see the
> ;; field `user-cancel-function' of the REGLOCK object if you need to
> ;; customize what to do on cancel.

It is not very clear how user can interact with "description". Is it a
tooltip? A window? Something else? Please give a bit more details.

Also, when "cancel" is requested, it is a good idea to state that
`user-cancel-function' is called and describe what it does by default.

> ;; When receiving the outcome (success or failure), after unlocking
> ;; the region, the library may leave information about the outcome

"may"?? Or does it always leave the information (by default)?

> ;; (using text properties/overlays).  If that outcome information is
> ;; (still) displayed, Emacs allows to display a description of that
> ;; lock.  From that description, the user may decide to "forget" that
> ;; lock; "forgetting the lock" removes the outcome visual marks, and,

Is "forgetting" customizeable like `user-cancel-function'?

> ;; it allows Emacs to discard any information related to this lock.

What does it mean?

> ;; The description of a lock (live or dead) provides information like
> ;; the schedule time, the duration, the outcome time, the result (in
> ;; case of success), the error (in case of failure), etc.  Customize
> ;; the field `insert-details-function' of REGLOCK object to add your
> ;; own information.

Please show an example how to do it here.

> ;; If the user kills a buffer, or, kills Emacs, some locks may have to
> ;; be killed too be killed too.  The library will ask the user to
 ^^^
typo

> ;; confirm if an operation requires to kill some locks.  See the field
> ;; `before-kill-function' of REGLOCK object, if you need to do
> ;; something before a lock is really killed.

Again, an example would be helpful.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Bruno Barbier]

Thanks for the review Ihor!

>> Thanks. I've improved the documentation of `org-pending' to mention
>> that one may want to customize the following fields of a reglock:
>> before-kill-function, user-cancel-function and
>> insert-details-function.  And, also, I added that one can attach
>> custom properties using the "properties" field.
>
> Thanks, but I still feel confused.
> May you:
>
> 1. Explain what does "kill" and "cancel" mean in the context of the
>REGLOCK.

I tried to document that as part of the function `org-pending. That is:

   |- Emacs may have to kill your locks; see the field
   |  `before-kill-function' if you wish to do something before your
   |  lock is killed.

In other words, Emacs has to kill the locks, because the user decided
to kill Emacs or some buffers.  The user has no say about this.  This
is when org-pending calls the private function
`org-pending--forced-kill'.
   
   
   |- The user may ask Emacs to cancel your lock; see the field
   |  `user-cancel-function' to override the default cancel function.

The user is not interested anymore by the lock.  He makes a polite
request to the library that locked the region to unlock it, the sooner
the better.

I added some documentation about them to the top level commentary.
   
> 2. Add information about visual hints, "kill", and "cancel" to the
>top-level commentary.
>
> For now, when reading the top commentary:
>
> ;; This library contains an API to lock a region while it is "being
> ;; updated"; the content of the region is "pending" and cannot be
> ;; modified.  It will be updated, later, when the new content is
> ;; available.
>
> I have an impression that the only side effect of "locking" is that the
> region becomes read-only. It is not clear at all that any other
> visual indication will be provided.

I added a description of the user interface at top-level.


[...]
> Yes, I do think that top commentary should explain this.
> The very idea that lock may be "canceled" or "killed" is important when
> designing Elisp code that uses the org-pending library.
>

Both "killed" and "canceled" are now described in the user interface
top-level section.


[...]
>
> I hope that the above clarifies what I am looking for.

I think it did.  And I even hope the improved top-level documentation
shows it :)


I've pushed the modification to my branch.

Thanks again for the review, your time and useful comments,


Bruno

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Ihor Radchenko]

Bruno Barbier  writes:

> Thanks for the review.
>
> I've pushed a new version, hoping to decrease the number of dislikes
> ;-)

Thanks!

>>> Cancel is handled by sending a failure message (see
>>>  `org-pending-cancel').  It's customizable using the reglock field
>>>  ~org-pending-reglock-user-cancel-function~, which can decide what to
>>>  do (like kill a process) and which can send a better outcome.
>>>  Standard 'cancel' leaves a failure outcome mark.
>>
>> Note that this function is not documented anywhere other than in reglock
>> class documentation.
>
> Thanks. I've improved the documentation of `org-pending' to mention
> that one may want to customize the following fields of a reglock:
> before-kill-function, user-cancel-function and
> insert-details-function.  And, also, I added that one can attach
> custom properties using the "properties" field.

Thanks, but I still feel confused.
May you:

1. Explain what does "kill" and "cancel" mean in the context of the
   REGLOCK.

2. Add information about visual hints, "kill", and "cancel" to the
   top-level commentary.

For now, when reading the top commentary:

;; This library contains an API to lock a region while it is "being
;; updated"; the content of the region is "pending" and cannot be
;; modified.  It will be updated, later, when the new content is
;; available.

I have an impression that the only side effect of "locking" is that the
region becomes read-only. It is not clear at all that any other
visual indication will be provided.

>> In general, I am confused about your overall design
>> of the user interaction with the locks.
>> The updated top commentary explains well how Elisp programs can send
>> data to the locks, but it does not say anything about how Elisp programs
>> can receive the data.
>
> An elisp program, that uses org-pending, must update the locks using
> `org-pending-send-update'.  That program does not receive any data
> from the lock; it may customize Emacs behavior using the reglock
> fields mentioned above: before-kill-function, user-cancel-function and
> insert-details-function.
>
> Hopefully, it's clearer now with the improved documentation of the
> org-pending function.
>
> Just let me know if you still think that the top commentary should
> explain this.  Thanks.

Yes, I do think that top commentary should explain this.
The very idea that lock may be "canceled" or "killed" is important when
designing Elisp code that uses the org-pending library.

>> Also, I'd like to see more information in the top commentary about what
>> are the "visual hints" displayed to the user and how to configure them.
>
> If you think the current "visual hints" are good enough and could be
> shipped as-is, in a first version (indirect buffers, etc.); I could
> work on documenting them better.  What kind of configuration are you
> thinking about ? just the faces ? or more advanced configurations ?

I am not thinking about details yet.
I just want insert-details-function to be described in the top
commentary. At high level. Something like


* Visual indication of the "pending" regions

While the region is locked, it is visually highlighted in the buffer,
providing information about the lock status as an overlay. The status
can be:

1. :scheduled - the region is "being updated", but the computation has
   not yet started.
2. :progress  - the computation is in progress

After unlocking the region, the visual indication is not necessarily
removed. Instead, the outcome is indicated in an overlay. The outcome
may be:

1. :success - the computation has been successful and the region text
   has been updated
2. :failure - the computation failed

The lock status is updated according to the data submitted to REGLOCK
object via `org-pending-send-update'.

* User interaction with pending regions

For any pending region, users may request detailed description to be
displayed. The description includes the pending region status, creation
time, outcome, duration, results, errors, etc.

Elisp code using this library may also supply additional details about
any given reglock, via `insert-details-function' field of REGLOCK
object. For example, process logs can be displayed this way.



I hope that the above clarifies what I am looking for.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Bruno Barbier]

Hi Ihor,

Thanks for the review.

I've pushed a new version, hoping to decrease the number of dislikes
;-)

Ihor Radchenko  writes:

> Bruno Barbier  writes:
>
>>> I have a further request on interaction with penreg objects.
>>> I feel that it is not ideal that overlays associated with penreg objects
>>> cannot be fully controlled by the callers.
>>
>> I'm trying to limit the public API surface.  I don't think we should
>> leak that we are currently using a mix of overlays and text
>> properties.
>
> Let me rephrase my concern - I do not like that after reglock is no
> longer live (got success/failure signal), there is no way to clean up the
> visual hints associated with this particular reglock.
[]

For the org-pending library, "live" means "locked".  Once the outcome
is known, it can't be "live" anymore (it's unlocked); as it's not
reusable, it's "dead".

As the region is not locked anymore, the lock properties/fields can't
be trusted anymore.

But see below about removing the visual outcome hints of a given
reglock.

>>> 2. Act on the outcome overlays - there is currently no way to remove
>>>them using penreg object.
>>
>> I've added a funcion `org-pending-delete-outcome-marks' to manually
>> delete outcome marks that are in a given region.
>>
>> Else, everything is handled automatically. Once the outcome is known,
>> the reglock is dead (not live-p).  org-pending may leave outcome marks
>> about the outcomes (outcome marks are optional).  The outcome marks
>> automatically disappear if the user remove the section, or, if a new
>> lock is created for the same region.
>
> I do not like this.
> I'd like the Elisp program that creates the reglock to be able to
> clean up any visual hints associated with it. > A function doing it for a
> given region cannot do this AFAIU.

ok. I've added the function `org-pending-reglock-delete-outcome-marks,
that will delete the outcome visual hints for a given reglock, if
there are some.

I updated how the lock is described to the user
(org-pending-describe-reglock): I added a button "Forget" (if the lock
is dead, that removes the outcome marks), and I added a "Cancel"
button if the lock is still live.


>>>Maybe :cancel signal? Canceled penreg
>>>objects can then be garbage-collected from the manager.
>>
>> Cancel is handled by sending a failure message (see
>>  `org-pending-cancel').  It's customizable using the reglock field
>>  ~org-pending-reglock-user-cancel-function~, which can decide what to
>>  do (like kill a process) and which can send a better outcome.
>>  Standard 'cancel' leaves a failure outcome mark.
>
> Note that this function is not documented anywhere other than in reglock
> class documentation.

Thanks. I've improved the documentation of `org-pending' to mention
that one may want to customize the following fields of a reglock:
before-kill-function, user-cancel-function and
insert-details-function.  And, also, I added that one can attach
custom properties using the "properties" field.

> In general, I am confused about your overall design
> of the user interaction with the locks.
> The updated top commentary explains well how Elisp programs can send
> data to the locks, but it does not say anything about how Elisp programs
> can receive the data.

An elisp program, that uses org-pending, must update the locks using
`org-pending-send-update'.  That program does not receive any data
from the lock; it may customize Emacs behavior using the reglock
fields mentioned above: before-kill-function, user-cancel-function and
insert-details-function.

Hopefully, it's clearer now with the improved documentation of the
org-pending function.

Just let me know if you still think that the top commentary should
explain this.  Thanks.


> Also, I'd like to see more information in the top commentary about what
> are the "visual hints" displayed to the user and how to configure them.

If you think the current "visual hints" are good enough and could be
shipped as-is, in a first version (indirect buffers, etc.); I could
work on documenting them better.  What kind of configuration are you
thinking about ? just the faces ? or more advanced configurations ?


[...]

>>> Is there any reason why you hide the extra information behind :-alist
>>> filed? Why not directly adding extra fields with proper documentation?
>>
>> To hide them, indeed :)
>
>> The API for 'get-status and 'get-live-p are
>> `org-pending-reglock-status' and `org-pending-reglock-live-p' (they
>> are read-only).  The API for the new `useless-p' is
>> `org-pending-reglock-useless-p' (it's read-only too).
>
> We usually "hide" fields by declaring them private.
> Hiding them from the type docs is not a good idea because it defeats the
> purpose of type documentation in general.
>
>> The fields anchor-ovl, region-ovl, on-outcome, set-status and
>> creation-point are the dump of the closure context, so that
>> org-pending doesn't rely anymore on a closure to handle updates; I've
>> rewritten that 

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Ihor Radchenko]

Bruno Barbier  writes:

>> I have a further request on interaction with penreg objects.
>> I feel that it is not ideal that overlays associated with penreg objects
>> cannot be fully controlled by the callers.
>
> I'm trying to limit the public API surface.  I don't think we should
> leak that we are currently using a mix of overlays and text
> properties.

Let me rephrase my concern - I do not like that after reglock is no
longer live (got success/failure signal), there is no way to clean up the
visual hints associated with this particular reglock.

>> In particular, I'd like to see some way to
>>
>> 1. Create penreg object without locking the region, so that scheduled-at
>>time is not set immediately and status overlay is not displayed.
>>Then, `org-pending-send-update' could send :schedule signal to
>>perform actual lock.
>
> Using the term "region" was confusing, sorry.  That's why I switched
> to region "lock".  I don't think there is a use to create a lock that
> doesn't lock.
>
> Also, that might be tricky to implement: `org-pending-send-update' is
> called asynchronously, from the user point of view.  Having regions
> that suddenly become locked, independently of what the user is
> currently doing (if we implement the :schedule message), might be
> difficult.
>
> What use do you have in mind ?

I was mostly confused about linkage between "visual hints" in buffer and
how they are connected with reglock object. You may discard this
comment of mine.

>> 2. Act on the outcome overlays - there is currently no way to remove
>>them using penreg object.
>
> I've added a funcion `org-pending-delete-outcome-marks' to manually
> delete outcome marks that are in a given region.
>
> Else, everything is handled automatically. Once the outcome is known,
> the reglock is dead (not live-p).  org-pending may leave outcome marks
> about the outcomes (outcome marks are optional).  The outcome marks
> automatically disappear if the user remove the section, or, if a new
> lock is created for the same region.

I do not like this.
I'd like the Elisp program that creates the reglock to be able to
clean up any visual hints associated with it. A function doing it for a
given region cannot do this AFAIU.

>>Maybe :cancel signal? Canceled penreg
>>objects can then be garbage-collected from the manager.
>
> Cancel is handled by sending a failure message (see
>  `org-pending-cancel').  It's customizable using the reglock field
>  ~org-pending-reglock-user-cancel-function~, which can decide what to
>  do (like kill a process) and which can send a better outcome.
>  Standard 'cancel' leaves a failure outcome mark.

Note that this function is not documented anywhere other than in reglock
class documentation. In general, I am confused about your overall design
of the user interaction with the locks.

The updated top commentary explains well how Elisp programs can send
data to the locks, but it does not say anything about how Elisp programs
can receive the data.

Also, I'd like to see more information in the top commentary about what
are the "visual hints" displayed to the user and how to configure them.
  
>> Also, the top-level commentary is getting incomplete and out-of-sync at
>> this point. May you work towards more detailed top-level description of
>> the library?
>
> Sorry. I tried hard to keep it in sync with all the modifications.
>
> I found it too much work, and, possibly overwhelming for the reader,
> to explain everything in the top-level "Commentary" section.
>
> That's why I deleted everything that wasn't mandatory to understand
> the core features.
>
> Everything should be documented as elisp documentation strings,
> following the documentation of `org-pending' and
> `org-pending-send-update', and, from code comments.

I agree, but I'd like to see the core concepts explained on top:
1. How to create region lock (done)
2. What the region lock does (prohibit modifications, done)
3. How the lock is presented to the user and how to control the presentation 
(not done)
4. What can user do with the lock and how it is reflected in Elisp level (not 
done)
5. What can Elisp do with the lock (done)

> I tried to improve the overall documentation.  I hope it's going to be
> easier for you, and others.

Yes! Thanks!

>> Is there any reason why you hide the extra information behind :-alist
>> filed? Why not directly adding extra fields with proper documentation?
>
> To hide them, indeed :)

> The API for 'get-status and 'get-live-p are
> `org-pending-reglock-status' and `org-pending-reglock-live-p' (they
> are read-only).  The API for the new `useless-p' is
> `org-pending-reglock-useless-p' (it's read-only too).

We usually "hide" fields by declaring them private.
Hiding them from the type docs is not a good idea because it defeats the
purpose of type documentation in general.

> The fields anchor-ovl, region-ovl, on-outcome, set-status and
> creation-point are the dump of the closure context, so that
> 

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Bruno Barbier]

Rudolf Adamkovič  writes:

>
> +1 for the full name.
>
> Searching 'M-x' for 'region' gives 229 results on my Emacs, so there is
> a precedent.  In fact, when I first read the name 'reglock', I took
> 'reg' for *not* a region, but register or registry, precisely because
> Emacs consistently spells the word out in full.
>

Thanks for your input.

Note that "reglock" is the current technical name, an implementation
detail, and for developer only.  The only current command using that
term (that shows up using M-x) is a debugging internal command
(org-pending--describe-reglock-at-point).

I'm thinking about using just "lock" but I'll keep using "reglock" for
now, as it's easier to search/replace, until the public API is
finalized.

Bruno

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Bruno Barbier]

Ihor Radchenko  writes:

> Bruno Barbier  writes:
>
>> I've pushed an update that should address most of your comments.
>
> Thanks!
>
>> I've found a better name: I'm now calling it a "lock".  So I renamed
>> "PENREG" into "REGLOCK" as "region lock".  The structure is now
>> `org-pending-reglock'.
>
> I slightly dislike short names and would prefer region-lock, but not a
> big deal - it is just my personal style preference.

It's more about being one word than being short, and being a new and
opaque word.  REGLOCK is a new technical term; it's definition is the
cl-defstruct.  For normal use, this is just a black box to pass
around.

I'll probably switch to using just "lock".



>> I've renamed org-pending-ti-send-update to org-pending-send-update
>> (now that the task control is gone, the prefix becomes useless).
>
> I have a further request on interaction with penreg objects.
> I feel that it is not ideal that overlays associated with penreg objects
> cannot be fully controlled by the callers.

I'm trying to limit the public API surface.  I don't think we should
leak that we are currently using a mix of overlays and text
properties.


> In particular, I'd like to see some way to
>
> 1. Create penreg object without locking the region, so that scheduled-at
>time is not set immediately and status overlay is not displayed.
>Then, `org-pending-send-update' could send :schedule signal to
>perform actual lock.

Using the term "region" was confusing, sorry.  That's why I switched
to region "lock".  I don't think there is a use to create a lock that
doesn't lock.

Also, that might be tricky to implement: `org-pending-send-update' is
called asynchronously, from the user point of view.  Having regions
that suddenly become locked, independently of what the user is
currently doing (if we implement the :schedule message), might be
difficult.

What use do you have in mind ?


> 2. Act on the outcome overlays - there is currently no way to remove
>them using penreg object.

I've added a funcion `org-pending-delete-outcome-marks' to manually
delete outcome marks that are in a given region.

Else, everything is handled automatically. Once the outcome is known,
the reglock is dead (not live-p).  org-pending may leave outcome marks
about the outcomes (outcome marks are optional).  The outcome marks
automatically disappear if the user remove the section, or, if a new
lock is created for the same region.


>Maybe :cancel signal? Canceled penreg
>objects can then be garbage-collected from the manager.

Cancel is handled by sending a failure message (see
 `org-pending-cancel').  It's customizable using the reglock field
 ~org-pending-reglock-user-cancel-function~, which can decide what to
 do (like kill a process) and which can send a better outcome.
 Standard 'cancel' leaves a failure outcome mark.
 
I've added garbage collections of useless reglocks (success or
failure): see `org-pending--mgr-garbage-collect'.



> Also, the top-level commentary is getting incomplete and out-of-sync at
> this point. May you work towards more detailed top-level description of
> the library?

Sorry. I tried hard to keep it in sync with all the modifications.

I found it too much work, and, possibly overwhelming for the reader,
to explain everything in the top-level "Commentary" section.

That's why I deleted everything that wasn't mandatory to understand
the core features.

Everything should be documented as elisp documentation strings,
following the documentation of `org-pending' and
`org-pending-send-update', and, from code comments.


> May you work towards more detailed top-level description of
> the library?
> This will make your ideas more explicit and make life
> easier for me during the further review (now, I have to guess what you
> meant by some parts of the code).

Sorry, and thank you again for your time.

I tried to improve the overall documentation.  I hope it's going to be
easier for you, and others.


>>> HANLDER will be another object we may expose via something like
>>> (org-pending-handler ( on-success-function on-cancel-function on-await 
>>> on-insert-logs) ...)
>>> Then, PENREG will call appropriate handler function as needed.
>>
>> As the task-control is now gone:
>>   - get/await is gone,
>>   - cancel is now a hook/function of REGLOCK,
>>   - insert-details is now a hook/function too of REGLOCK.
>>> ...
>>> Yes. Lexical context is implicit and harder to debug, while storing
>>> necessary data explicitly in the struct slots is more robust as we are
>>> very clear which context is intended to be captured.
>>
>> Done.
>
> Is there any reason why you hide the extra information behind :-alist
> filed? Why not directly adding extra fields with proper documentation?

To hide them, indeed :)

The API for 'get-status and 'get-live-p are
`org-pending-reglock-status' and `org-pending-reglock-live-p' (they
are read-only).  The API for the new `useless-p' is
`org-pending-reglock-useless-p' (it's read-only 

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Rudolf Adamkovič]

Ihor Radchenko  writes:

>> I've found a better name: I'm now calling it a "lock".  So I renamed
>> "PENREG" into "REGLOCK" as "region lock".  The structure is now
>> `org-pending-reglock'.
>
> I slightly dislike short names and would prefer region-lock, but not a
> big deal - it is just my personal style preference.

+1 for the full name.

Searching 'M-x' for 'region' gives 229 results on my Emacs, so there is
a precedent.  In fact, when I first read the name 'reglock', I took
'reg' for *not* a region, but register or registry, precisely because
Emacs consistently spells the word out in full.

Rudy
-- 
"I love deadlines.  I love the whooshing noise they make as they go by."
--- Douglas Adams, The Salmon of Doubt, 2002

Rudolf Adamkovič  [he/him]
Studenohorská 25, 84103 Bratislava, Slovakia, European Union

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Ihor Radchenko]

Bruno Barbier  writes:

> I've pushed an update that should address most of your comments.

Thanks!

> I've found a better name: I'm now calling it a "lock".  So I renamed
> "PENREG" into "REGLOCK" as "region lock".  The structure is now
> `org-pending-reglock'.

I slightly dislike short names and would prefer region-lock, but not a
big deal - it is just my personal style preference.

> I've renamed org-pending-ti-send-update to org-pending-send-update
> (now that the task control is gone, the prefix becomes useless).

I have a further request on interaction with penreg objects.
I feel that it is not ideal that overlays associated with penreg objects
cannot be fully controlled by the callers.

In particular, I'd like to see some way to

1. Create penreg object without locking the region, so that scheduled-at
   time is not set immediately and status overlay is not displayed.
   Then, `org-pending-send-update' could send :schedule signal to
   perform actual lock.

2. Act on the outcome overlays - there is currently no way to remove
   them using penreg object. Maybe :cancel signal? Canceled penreg
   objects can then be garbage-collected from the manager.

Also, the top-level commentary is getting incomplete and out-of-sync at
this point. May you work towards more detailed top-level description of
the library? This will make your ideas more explicit and make life
easier for me during the further review (now, I have to guess what you
meant by some parts of the code).

>> HANLDER will be another object we may expose via something like
>> (org-pending-handler ( on-success-function on-cancel-function on-await 
>> on-insert-logs) ...)
>> Then, PENREG will call appropriate handler function as needed.
>
> As the task-control is now gone:
>   - get/await is gone,
>   - cancel is now a hook/function of REGLOCK,
>   - insert-details is now a hook/function too of REGLOCK.
>> ...
>> Yes. Lexical context is implicit and harder to debug, while storing
>> necessary data explicitly in the struct slots is more robust as we are
>> very clear which context is intended to be captured.
>
> Done.

Is there any reason why you hide the extra information behind :-alist
filed? Why not directly adding extra fields with proper documentation?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Bruno Barbier]

Hi Ihor,

Thanks for your review and detailed explanations.

I've pushed an update that should address most of your comments.

Let me answer point by point below.

Ihor Radchenko  writes:

> Bruno Barbier  writes:
>
>>> I feel that org-pending-penreg (org-pending-) is
>>> redundant. Maybe better use org-pending-region?
>>
>> PENREG is the name of the structure; the "org-pending" is the
>> mandatory library prefix; this mechanically gives the name.  A PENDREG
>> object is not a "region" in Emacs sense.
>>
>> Do you see a better name for the structure PENREG, so that it doesn't
>> sound like a "region" ?
>
> Library prefix is also a part of the name and delivers useful
> information. "org-pending-region" and "region" and not the same names.
>
> We make use of prefix semantic in various places:
> - org-export-backend, implying not just "backend", but also "export"
> - org-cite-processor, implying not just "processor", but also "cite"
> - org-lint-checker - "org-lint" + "checker"
> - org-element-deferred - "org-element" + "deferred"
>
> So, there is no need to duplicate information from the prefix - it is an
> integral part of the struct name. Doing otherwise would go again the
> existing naming in Org code base.

Got it. Thanks for the explanation.

I've found a better name: I'm now calling it a "lock".  So I renamed
"PENREG" into "REGLOCK" as "region lock".  The structure is now
`org-pending-reglock'.


>>> 1. It is not clear why you need a separate ~virtual~ field. When
>>>~region~ is nil it already implies that the pending region is
>>>virtual.
>>
[...]
> Also, ~virtual~ field is unused. So, maybe we can even drop it
> completely. We can always add new fields in future, if a need arises.

The region is not allowed to be nil anymore. All "virtual" issues
are solved! :)


>>> 3. ~source~ field must match the ~region~ marker buffer. Then, why do we
>>>need this field at all? May as well just use (marker-buffer (car region))
>>
>> The "source" is the region requesting the update.
>
> The docstring of `org-pending' states that it is a buffer position:
>
> The SOURCE is the buffer position that requested this pending region.

Sorry. It was, yes.

>> ... The pending region
>> is the "target" of the update, i.e. the region that will be updated.
>>
>>
>> For example, in DEMO_ONLY, with org-babel, these 2 regions are never
>> the same:
>>1. the source is the source code block,
>>2. the target (pending region) is the result region.
>
> I am wondering why source must be a buffer position.
> What if we want to mark a region pending for some task not associated
> with a source? And why do we need to know the source at all?

Good point. It was to allow the user to quickly jump to the source
code block; I removed it from org-pending.



>>  2. insert-details: If, and only if, the user decides to
>>  investigate what happened, Emacs will ask the task if it has any
>>  details to add, that might help the user (like exit-code for an
>>  OS process, stderr for an OS process or link to a log file, etc.)
>
> I have to say that I am confused about "insert-details" part. Mostly
> because it is not per se connected to the associated task. It is rather
> an additional handler used to provide debug information about the task
> status and outcome.
> AFAIU, it is conceptually very similar to HANDLE-RESULT function.

You're right: it was confusing.  It's now like a hook, that
org-pending-describe-reglock will use.  It should now be clearer that
it's for "information purposes" only.


> I think that rather than handing HANDLE-RESULT and also TASK-CONTROL, we
> may reduce everything to a single "handler" object that will serve as a
> way for PENREG to communicate back to Elisp. That way, we do not need to
> have a concept of a "task".

I removed the task-control, and, the concept of "task".  HANDLE-RESULT
is gone from org-pending.  `org-pending' has a new keyword: :on-outcome
that will allow to do anything, both on success and on failure.

> Instead, it will be a familiar async API
> with ability to (1) create (2) send signals to (3) receive signals from
> PENREG object.
> `org-pending' will be the entry point to create PENREG object.
>
> `org-pending-ti-send-update' (or maybe simply
> `org-pending-send-update'?) will be a way to send data to PENREG object.
>

I've renamed org-pending-ti-send-update to org-pending-send-update
(now that the task control is gone, the prefix becomes useless).


> HANLDER will be another object we may expose via something like
> (org-pending-handler ( on-success-function on-cancel-function on-await 
> on-insert-logs) ...)
> Then, PENREG will call appropriate handler function as needed.

As the task-control is now gone:
  - get/await is gone,
  - cancel is now a hook/function of REGLOCK,
  - insert-details is now a hook/function too of REGLOCK.


>>>If the argument to ~org-pending-task-connect~ is a lambda, we can use
>>>the current approach you implemented 

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Ihor Radchenko]

Bruno Barbier  writes:

>> I feel that org-pending-penreg (org-pending-) is
>> redundant. Maybe better use org-pending-region?
>
> PENREG is the name of the structure; the "org-pending" is the
> mandatory library prefix; this mechanically gives the name.  A PENDREG
> object is not a "region" in Emacs sense.
>
> Do you see a better name for the structure PENREG, so that it doesn't
> sound like a "region" ?

Library prefix is also a part of the name and delivers useful
information. "org-pending-region" and "region" and not the same names.

We make use of prefix semantic in various places:
- org-export-backend, implying not just "backend", but also "export"
- org-cite-processor, implying not just "processor", but also "cite"
- org-lint-checker - "org-lint" + "checker"
- org-element-deferred - "org-element" + "deferred"

So, there is no need to duplicate information from the prefix - it is an
integral part of the struct name. Doing otherwise would go again the
existing naming in Org code base.

>> 1. It is not clear why you need a separate ~virtual~ field. When
>>~region~ is nil it already implies that the pending region is
>>virtual.
>
> It's a constant.  Calling a function looks more like we need to
> recompute it each time, and, we could even change it.  And
> cl-defstruct writes the function for us.
>
> Do you prefer a manually written function ?

Either a function or a clear indication in the docstring that ~virtual~
and ~region~ are connected and both read-only.

Also, ~virtual~ field is unused. So, maybe we can even drop it
completely. We can always add new fields in future, if a need arises.

>> 3. ~source~ field must match the ~region~ marker buffer. Then, why do we
>>need this field at all? May as well just use (marker-buffer (car region))
>
> The "source" is the region requesting the update.

The docstring of `org-pending' states that it is a buffer position:

The SOURCE is the buffer position that requested this pending region.

> ... The pending region
> is the "target" of the update, i.e. the region that will be updated.
>
>
> For example, in DEMO_ONLY, with org-babel, these 2 regions are never
> the same:
>1. the source is the source code block,
>2. the target (pending region) is the result region.

I am wondering why source must be a buffer position.
What if we want to mark a region pending for some task not associated
with a source? And why do we need to know the source at all?

>  2. insert-details: If, and only if, the user decides to
>  investigate what happened, Emacs will ask the task if it has any
>  details to add, that might help the user (like exit-code for an
>  OS process, stderr for an OS process or link to a log file, etc.)

I have to say that I am confused about "insert-details" part. Mostly
because it is not per se connected to the associated task. It is rather
an additional handler used to provide debug information about the task
status and outcome.

AFAIU, it is conceptually very similar to HANDLE-RESULT function.

I think that rather than handing HANDLE-RESULT and also TASK-CONTROL, we
may reduce everything to a single "handler" object that will serve as a
way for PENREG to communicate back to Elisp. That way, we do not need to
have a concept of a "task". Instead, it will be a familiar async API
with ability to (1) create (2) send signals to (3) receive signals from
PENREG object.

`org-pending' will be the entry point to create PENREG object.

`org-pending-ti-send-update' (or maybe simply
`org-pending-send-update'?) will be a way to send data to PENREG object.

HANLDER will be another object we may expose via something like
(org-pending-handler ( on-success-function on-cancel-function on-await 
on-insert-logs) ...)
Then, PENREG will call appropriate handler function as needed.

>>If the argument to ~org-pending-task-connect~ is a lambda, we can use
>>the current approach you implemented on the branch.
>
>> 2. ~org-pending-task-send-update~ name is confusing - it reads as if we
>>send an update _to_ the task. Maybe better ~org-pending-region-update~?
>
> Yes ... I wanted a common prefix for the 3 functions that a "task"
> implementation is allowed to use:
> - org-pending-task-connect,
> - org-pending-task-send-update,
> - org-pending-task-not-implemented.
>
> It's not confusing if one ignores the common prefix :-)
>
> I've renamed all these functions from "org-pending-task-" to
> "org-pending-ti-" where "ti" stands for "task implementation".

I still feel confused. As stated above, it might be a good idea to get
rid of the concept of "task" completely.

>>Then, we might even drop ~-sentinel~ field in org-pending-penreg
>>object and instead implement that hard-coded ~update~ lambda from
>>~org-pending~ as a part of ~org-pending-region-update~.
>
> That would require to manually capture (dump/load) the context that
> the sentinel closure is automatically capturing.
>
> Why would it be better ? 

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Bruno Barbier]

Ihor Radchenko  writes:

Thanks for your review Ihor!

> Bruno Barbier  writes:
>
>> I rewrote the API, rename many things, moved the code around and
>> sorted everything into heading/subheading sections.  This is hopefully
>> less confusing and a lot simpler; and the documentation hopefully
>> explains better how to use it.
>
> Thanks! It does look more clear.
>
>> The updated section "Commentary", in org-pending, describes the 3 steps
>> that are needed to mark and use a pending region: a PENREG (I've renamed
>> "PINFO" to "PENREG", for PENding REGion, more specific).
>
> I feel that org-pending-penreg (org-pending-) is
> redundant. Maybe better use org-pending-region?

PENREG is the name of the structure; the "org-pending" is the
mandatory library prefix; this mechanically gives the name.  A PENDREG
object is not a "region" in Emacs sense.

Do you see a better name for the structure PENREG, so that it doesn't
sound like a "region" ?


>> WDYT of this version ?
>
> Comments on org-pending-pendreg struct:
>
> 1. It is not clear why you need a separate ~virtual~ field. When
>~region~ is nil it already implies that the pending region is
>virtual.

It's a constant.  Calling a function looks more like we need to
recompute it each time, and, we could even change it.  And
cl-defstruct writes the function for us.

Do you prefer a manually written function ?


> 2. ~id~ field is semi-internal and assumed to be a number.
>Maybe we can do something more similar to Emacs process API:
>
>(make-process  ARGS)
>...
>:name NAME -- NAME is name for process.  It is modified if necessary
>to make it unique.
>
>We can replace id with human-readable name that can also be supplied
>when creating a pending region

Good idea. Done.


> 3. ~source~ field must match the ~region~ marker buffer. Then, why do we
>need this field at all? May as well just use (marker-buffer (car region))

The "source" is the region requesting the update.  The pending region
is the "target" of the update, i.e. the region that will be updated.


For example, in DEMO_ONLY, with org-babel, these 2 regions are never
the same:
   1. the source is the source code block,
   2. the target (pending region) is the result region.

I tried to improve the documentation of `org-pending-penreg' (source &
region).


> On the design of ~org-pending~ and ~org-pending-task-connect~:
>
> 1. I feel confused about the overall design of the interaction between
>pending region and the associated task.
>
>Functions like ~org-pending-task-send-update~ imply that pending
>region is rather decoupled from from associated task and the task
>should arrange manually for sending updates to the pending region
>object.

Exactly: the task implementation must use these ~org-pending-task-XXX~
functions to send updates to one (or more) pending region(s).


>On the other hand, there is ~task-connection~ that is used to kill
>associated task/process or to "await" for it. This time, pending
>region is strongly coupled with the task, killing it upon deleting
>the pending region.


These are optional features; and only ~org-pending~ will know if and
when those might be useful.  That's why the task needs to provide
callbacks here.
 1. cancel: Emacs may, in exceptional cases only,
 send a "cancel" to the task, meaning, "The user destroyed the
 pending region, and thus, Emacs will not use any update for it".

 2. insert-details: If, and only if, the user decides to
 investigate what happened, Emacs will ask the task if it has any
 details to add, that might help the user (like exit-code for an
 OS process, stderr for an OS process or link to a log file, etc.)

 3. get (await): It's an (unofficial) way, (in the degenerate case
 where the task implementation gives up on asynchronicity) to
 block until the outcome is available.  `org-pending' itself
 doesn't use it; DEMO_ONLY uses it with org-babel to define the
 synchronous executions.

>I think that we need more (optional) coupling between pending region
>and the associated task. We should be able to get more information
>about the task from pending region side - get logs, current status,
>exit status, etc.


>More specifically, I think that we need (1) allow to pass task as an
>argument for ~org-pending~.

That's actually what I started with, but, I couldn't make it work.

Breaking it like this is what allowed me to get the most generic and
simplest API that works for anything: threads, callbacks, OS processes,
timers.

If org-pending takes a "task" as an argument, then, we have to define
a universal API for whatever a "task" might be: threads, processes,
callbacks, timers, etc. and any combination of them.

It looks simpler to say that the "task" (whatever "task" means), MAY
call:
- org-pending-task-send-update (:progress xxx1)
- org-pending-task-send-update (:progress xxx2)
- 

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Ihor Radchenko]

Bruno Barbier  writes:

> I rewrote the API, rename many things, moved the code around and
> sorted everything into heading/subheading sections.  This is hopefully
> less confusing and a lot simpler; and the documentation hopefully
> explains better how to use it.

Thanks! It does look more clear.

> The updated section "Commentary", in org-pending, describes the 3 steps
> that are needed to mark and use a pending region: a PENREG (I've renamed
> "PINFO" to "PENREG", for PENding REGion, more specific).

I feel that org-pending-penreg (org-pending-) is
redundant. Maybe better use org-pending-region?

> WDYT of this version ?

Comments on org-pending-pendreg struct:

1. It is not clear why you need a separate ~virtual~ field. When
   ~region~ is nil it already implies that the pending region is
   virtual.

2. ~id~ field is semi-internal and assumed to be a number.
   Maybe we can do something more similar to Emacs process API:
   
   (make-process  ARGS)
   ...
   :name NAME -- NAME is name for process.  It is modified if necessary
   to make it unique.

   We can replace id with human-readable name that can also be supplied
   when creating a pending region

3. ~source~ field must match the ~region~ marker buffer. Then, why do we
   need this field at all? May as well just use (marker-buffer (car region))

On the design of ~org-pending~ and ~org-pending-task-connect~:

1. I feel confused about the overall design of the interaction between
   pending region and the associated task.

   Functions like ~org-pending-task-send-update~ imply that pending
   region is rather decoupled from from associated task and the task
   should arrange manually for sending updates to the pending region
   object.

   On the other hand, there is ~task-connection~ that is used to kill
   associated task/process or to "await" for it. This time, pending
   region is strongly coupled with the task, killing it upon deleting
   the pending region.

   I think that we need more (optional) coupling between pending region
   and the associated task. We should be able to get more information
   about the task from pending region side - get logs, current status,
   exit status, etc.

   More specifically, I think that we need (1) allow to pass task as an
   argument for ~org-pending~. (2) In ~org-pending-task-connect~, we
   should allow the task to be a process or timer object. Then, we can
   automatically arrange retrieving process/timer status from the task:
   
   Use process sentinel (maybe, modifying existing via ~add-function~)
   to arrange process status changes to be automatically submitted to
   the pending region;

   Get log updates via process filter

   Kill process via ~kill-process~

   Similar for timers.

   If the argument to ~org-pending-task-connect~ is a lambda, we can use
   the current approach you implemented on the branch.

2. ~org-pending-task-send-update~ name is confusing - it reads as if we
   send an update _to_ the task. Maybe better ~org-pending-region-update~?
   Then, we might even drop ~-sentinel~ field in org-pending-penreg
   object and instead implement that hard-coded ~update~ lambda from
   ~org-pending~ as a part of ~org-pending-region-update~.

3. I feel that different handling of "owner" and indirect buffers is not
   right.

   From the user perspective, it does not matter whether we run an src
   block from one or another indirect buffers - it makes sense to see
   the status in all the indirect org-mode buffers. Maybe we can hook
   into org-mode's fontification and display pending overlays in all the
   indirect buffers.

   Further, it is very confusing that running src block twice from the
   same buffer is not the same as running the same src block from one
   buffer and then from another indirect buffer. The current
   implementation of ~remove-previous-overlays~ makes such distinction
   for reasons I do not understand.

4. I have questions about ~handle-result~ argument in ~org-pending~.

   It is only called on success, but I can easily see that we need to
   handle things specially on failure as well. For example, insert
   stderr or perform other actions like displaying some buffer.

   Or we may even hook some special action to clicking on status
   overlay. For example, clicking on "failure" status overlay may raise
   stderr log.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Bruno Barbier]

Ihor Radchenko  writes:

> Bruno Barbier  writes:
>> Would "lisp/org-pending.el" be OK ?  Or do you see a better place/name ?
>
> org-pending sounds fine. Maybe org-pending-text to be more explicit.

I've picked: "org-pending"; it's shorter and can by used as a prefix.
Thanks.


> Similar to process API, your library provides means to interact with
> process associated with the pending text - create them (40.4 Creating an
> Asynchronous Process), handle feedback (40.9.2 Process Filter
> Functions), handle on-done/fail/etc (40.10 Sentinels: Detecting Process
> Status Changes), kill/cancel execution (40.5 Deleting Processes, 40.8
> Sending Signals to Processes), insert-log (strerr in Emacs processes)
> list pending (40.6 Process Information), org-pending-on-kill-buffer
> (40.11 Querying Before Exit), get information (`process-attributes',
> 40.12 Accessing Other Processes)
>
> I simply went across 40 Processes section of Elisp manual and I see
> parallels for pretty much every subsection; but the terminology _and
> API_ are different.
>

I rewrote the API, rename many things, moved the code around and
sorted everything into heading/subheading sections.  This is hopefully
less confusing and a lot simpler; and the documentation hopefully
explains better how to use it.

The updated section "Commentary", in org-pending, describes the 3 steps
that are needed to mark and use a pending region: a PENREG (I've renamed
"PINFO" to "PENREG", for PENding REGion, more specific).

I tried to stick with the process terminology when relevant (mostly
kill-query and sentinel).

It doesn't really match the process API, but, now, it should be easier to
understand why.


>> I'm OK to rewrite it using a cl-defstruct (to be honnest, I just
>> flipped a coin to decide between a struct or an alist :) ).
>
> I think that I need to elaborate.
> It is not necessarily a matter of alist vs. struct, but more about the
> API. I do not like the idea of mutating the alist entries directly. I'd
> rather expose an API to work with "pending" as an opaque object - send
> signals to it, get status, etc.
>
> struct is slightly better here because it automatically defines setters
> and getters for all the slots without a need to write extra boilerplate
> code.

Done. Thanks!

I added a function `org-pending-user-edit': it allows to edit a region
using org-pending and `string-edit' (see org-pending.el).

I also added an other execution engine (DEMO_ONLY), that allows to
execute any elisp asynchronously, using callbacks (see
`my-use-callbacks-schedule', in my-async-tests.el), and in
my-async-tests.org, near the multithreading examples.

To test, follow the org file [my-async-tests.org] (see direct link below).

WDYT of this version ?

Thanks!

Bruno

[I've forced push the new version, squashing everything into simple
commits, if you need the old version, just ask and I'll push a copy
there.]


[my-async-tests.org]:
 
https://framagit.org/brubar/org-mode-mirror/-/blob/bba-pending-contents/scratch/bba-pending-contents/my-async-tests.org?ref_type=heads

[repo & branch]:
 
https://framagit.org/brubar/org-mode-mirror/-/tree/bba-pending-contents?ref_type=heads
 bba-pending-contents

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Ihor Radchenko]

Bruno Barbier  writes:

>> While reading the library header and `org-pending' docstring (btw, it
>> should probably be a separate library, not a part of org-macs),
>
> I was feeling more and more like squatting the wrong file :-)
>
> Would "lisp/org-pending.el" be OK ?  Or do you see a better place/name ?

org-pending sounds fine. Maybe org-pending-text to be more explicit.

>> More or less, org-pending implements Elisp asynchronous process control,
>> but the processes are associated with region, not the whole buffer.
>> Hence, I feel that we should adopt terminology similar to the existing
>> terminology for processes - process filters, process sentinels, sending
>> signals to processes, etc. And similar API.
>
> The current `org-pending' doesn't know what a process is, or a thread.
> The job of org-pending is only to let Emacs know that a given region
> will be updated at some point later (much like what
> `org-edit-src-code' does, when a source block is being edited
> manually).

> I've re-read the Elisp manual about asynchronous processes.  The only
> thing that seemed obvious to me, was to replace "feedbacks-handler"
> with "sentinel".  I also tried to simplify things and use REGION
> instead of CONTENT where it make sense.  Thanks.

> Do you see something else ?

Similar to process API, your library provides means to interact with
process associated with the pending text - create them (40.4 Creating an
Asynchronous Process), handle feedback (40.9.2 Process Filter
Functions), handle on-done/fail/etc (40.10 Sentinels: Detecting Process
Status Changes), kill/cancel execution (40.5 Deleting Processes, 40.8
Sending Signals to Processes), insert-log (strerr in Emacs processes)
list pending (40.6 Process Information), org-pending-on-kill-buffer
(40.11 Querying Before Exit), get information (`process-attributes',
40.12 Accessing Other Processes)

I simply went across 40 Processes section of Elisp manual and I see
parallels for pretty much every subsection; but the terminology _and
API_ are different.

>> Also, I am not sure if I like the idea of exposing raw PINFO alist and
>> mutating it. In particular, I have doubts about mutating CONTENT.
>> What we might do instead is implement PINFO as struct with custom
>> accessors/setters.
>
> I went for this simple alist because it's more flexible (easy to add new
> bindings) and less code than using a struct.  And Org already uses lists
> like this (params, info, etc.).  Mutating CONTENT (or other essential
> data) would be a very bad idea indeed; but it's like that for pretty
> much everything with elisp.
>
> I'm OK to rewrite it using a cl-defstruct (to be honnest, I just
> flipped a coin to decide between a struct or an alist :) ).

I think that I need to elaborate.
It is not necessarily a matter of alist vs. struct, but more about the
API. I do not like the idea of mutating the alist entries directly. I'd
rather expose an API to work with "pending" as an opaque object - send
signals to it, get status, etc.

struct is slightly better here because it automatically defines setters
and getters for all the slots without a need to write extra boilerplate
code.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Bruno Barbier]

Hi Ihor,

Ihor Radchenko  writes:

> Thanks!
> I have some minor concerns about implementation, but you clearly
> demonstrated the things can be working in general.

Thanks!

> While reading the library header and `org-pending' docstring (btw, it
> should probably be a separate library, not a part of org-macs),

I was feeling more and more like squatting the wrong file :-)

Would "lisp/org-pending.el" be OK ?  Or do you see a better place/name ?


> I felt confused about terminology and also had déjà vu's about what
> org-pending does.

> More or less, org-pending implements Elisp asynchronous process control,
> but the processes are associated with region, not the whole buffer.
> Hence, I feel that we should adopt terminology similar to the existing
> terminology for processes - process filters, process sentinels, sending
> signals to processes, etc. And similar API.

The current `org-pending' doesn't know what a process is, or a thread.
The job of org-pending is only to let Emacs know that a given region
will be updated at some point later (much like what
`org-edit-src-code' does, when a source block is being edited
manually).

I've re-read the Elisp manual about asynchronous processes.  The only
thing that seemed obvious to me, was to replace "feedbacks-handler"
with "sentinel".  I also tried to simplify things and use REGION
instead of CONTENT where it make sense.  Thanks.

Do you see something else ?



> Also, I am not sure if I like the idea of exposing raw PINFO alist and
> mutating it. In particular, I have doubts about mutating CONTENT.
> What we might do instead is implement PINFO as struct with custom
> accessors/setters.

I went for this simple alist because it's more flexible (easy to add new
bindings) and less code than using a struct.  And Org already uses lists
like this (params, info, etc.).  Mutating CONTENT (or other essential
data) would be a very bad idea indeed; but it's like that for pretty
much everything with elisp.

I'm OK to rewrite it using a cl-defstruct (to be honnest, I just
flipped a coin to decide between a struct or an alist :) ).

I'll push my new version after relocating everything into its own
file. Let me know if lisp/org-pending.el is OK.

Thanks.

Bruno

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Ihor Radchenko]

Bruno Barbier  writes:

>> I now think that overlays are the right way; the /pending content/ is
>> attached to one buffer: a base or a clone; this is for the user to
>> decide.
>>
>> I will manually add text properties, below the overlay, to mark the text
>> as /pending/, so that pending contents will be visible and read-only in
>> all other buffers, base or indirect ones.

Thanks!
I have some minor concerns about implementation, but you clearly
demonstrated the things can be working in general.

Let me now zoom out all the way down to org-pending library design.

While reading the library header and `org-pending' docstring (btw, it
should probably be a separate library, not a part of org-macs), I felt
confused about terminology and also had déjà vu's about what org-pending
does.

More or less, org-pending implements Elisp asynchronous process control,
but the processes are associated with region, not the whole buffer.
Hence, I feel that we should adopt terminology similar to the existing
terminology for processes - process filters, process sentinels, sending
signals to processes, etc. And similar API.

Also, I am not sure if I like the idea of exposing raw PINFO alist and
mutating it. In particular, I have doubts about mutating CONTENT.
What we might do instead is implement PINFO as struct with custom
accessors/setters.

WDYT?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Bruno Barbier]

Hi,

Bruno Barbier  writes:

> Ihor Radchenko  writes:
>
>> Bruno Barbier  writes:
>>
 Overlays are not transferred when a new indirect buffer is created (for
 example, by org-capture, or by user). So, it will be (1) impossible to
 see pending overlays in indirect buffers; (2) user edits of pending text
 from indirect buffer will not be prevented.
>>>
[...]
> I now think that overlays are the right way; the /pending content/ is
> attached to one buffer: a base or a clone; this is for the user to
> decide.
>
> I will manually add text properties, below the overlay, to mark the text
> as /pending/, so that pending contents will be visible and read-only in
> all other buffers, base or indirect ones.
>
> Cloning buffers is easy to test. I'm not sure which scenario I should
> use to test org-capture though.
>
> I'll update my branch with that improvement soon.
>
>
> Thanks Ihor!
>
> Bruno



Hi,

After some work, some bug fixes and a few segfaults (using indirect
buffers, see bug#69529), I pushed a new version.  The main changes are:

• Handle indirect buffers: the pending content belongs to the buffer
  that started it (using text properties to mirror overlays).
• Use the fringe to indicate success or failure.
• Describe the pending content (past) when the user clicks it
  (pending, success or failure, time, duration, log, etc.).
• Improve the logging API and provide examples (may be used to collect
  stderr for example).

As before, the org file describes how to test it, see the file
[scratch/bba-pending-contents/my-async-tests.org] (direct link below).

Comments, critiques, ideas, corrections are most welcome.

Thanks,

Bruno


[scratch/bba-pending-contents/my-async-tests.org]

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Bruno Barbier]

Ihor Radchenko  writes:

> Bruno Barbier  writes:
>
>>> Overlays are not transferred when a new indirect buffer is created (for
>>> example, by org-capture, or by user). So, it will be (1) impossible to
>>> see pending overlays in indirect buffers; (2) user edits of pending text
>>> from indirect buffer will not be prevented.
>>
>> If I understand correctly, you would prefer a solution that relies on
>> text properties ? (I didn't leak the overlay details in the API, so, it
>> should not be too hard to switch to text properties, except for possible
>> conflicts with other existing properties).
>
> Yes, I think. Might also try to mirror overlays, but that's problematic
> in practice.

I now think that overlays are the right way; the /pending content/ is
attached to one buffer: a base or a clone; this is for the user to
decide.

I will manually add text properties, below the overlay, to mark the text
as /pending/, so that pending contents will be visible and read-only in
all other buffers, base or indirect ones.

Cloning buffers is easy to test. I'm not sure which scenario I should
use to test org-capture though.

I'll update my branch with that improvement soon.


Thanks Ihor!

Bruno

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Ihor Radchenko]

Bruno Barbier  writes:

>> Overlays are not transferred when a new indirect buffer is created (for
>> example, by org-capture, or by user). So, it will be (1) impossible to
>> see pending overlays in indirect buffers; (2) user edits of pending text
>> from indirect buffer will not be prevented.
>
> If I understand correctly, you would prefer a solution that relies on
> text properties ? (I didn't leak the overlay details in the API, so, it
> should not be too hard to switch to text properties, except for possible
> conflicts with other existing properties).

Yes, I think. Might also try to mirror overlays, but that's problematic
in practice.

However, you cannot use 'before-string/'after-string for overlays.

> What about markers ? Are they OK ?

Markers always point to a specific buffer.
However, for indirect buffers, the text and text properties are shared.
So, editing in one buffer will be automatically reflected in all the clones.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Bruno Barbier]

Hi Ihor,


Ihor Radchenko  writes:

[...]
>> I've tried to fully describe the feature in the new section "Pending
>> contents", in the file `lisp/org-macs.el'.
>
> I have one general concern about the implementation.
>
> Overlays are not transferred when a new indirect buffer is created (for
> example, by org-capture, or by user). So, it will be (1) impossible to
> see pending overlays in indirect buffers; (2) user edits of pending text
> from indirect buffer will not be prevented.

If I understand correctly, you would prefer a solution that relies on
text properties ? (I didn't leak the overlay details in the API, so, it
should not be too hard to switch to text properties, except for possible
conflicts with other existing properties).

What about markers ? Are they OK ?

Thanks,


Bruno

Re: Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Ihor Radchenko]

Bruno Barbier  writes:

> I've publish the proposed changes as a branch.  You can fecth that
> branch there:
> ┌
> │ repo:   https://framagit.org/brubar/org-mode-mirror
> │ branch: bba-pending-contents
> └

Thanks!

> I've tried to fully describe the feature in the new section "Pending
> contents", in the file `lisp/org-macs.el'.

I have one general concern about the implementation.

Overlays are not transferred when a new indirect buffer is created (for
example, by org-capture, or by user). So, it will be (1) impossible to
see pending overlays in indirect buffers; (2) user edits of pending text
from indirect buffer will not be prevented.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Pending contents in org documents (Re: Asynchronous blocks for everything (was Re: ...))

[Bruno Barbier]

Hi,

Bruno Barbier  writes:

> Hi Matt,
[...]
>> Since this thread is dedicated to blocking, let me share my thoughts on that 
>> subject.
>
> I guess I should start a new thread then :)


I finally got a new version.  I've renamed the proposed feature "pending
contents", that is, some contents that something will update *later*.
With that feature, source code blocks and dynamic blocks can be made
asynchronous … and more!

I've publish the proposed changes as a branch.  You can fecth that
branch there:
┌
│ repo:   https://framagit.org/brubar/org-mode-mirror
│ branch: bba-pending-contents
└

The file [my-async-tests.org] describes the proposed changes, its
implementation and contains examples to play with it (shell, python,
ruby, results{append,prepend,silent}, inline blocks, multithreading,
dynamic asynchronous blocks, source codes that depend on other blocks
{post, stdin}, etc.).

I've tried to fully describe the feature in the new section "Pending
contents", in the file `lisp/org-macs.el'.

Testing it in a bare Emacs should now work (I'm using 30.0.50).

*DO NOT TEST* in your production Emacs: this is *alpha* software and it
hooks itself deeply into Emacs (kill hooks), and (only when trying the
multithread examples) uses thread mutexes

Here is a simple recipe to test the proposed "pending contents" feature:
┌
│ git clone -b bba-pending-contents https://framagit.org/brubar/org-mode-mirror
│ cd org-mode-mirror
│ make compile
│ cd scratch/bba-pending-contents/
│ emacs -q -L ../../lisp my-async-tests.org
└


Thanks you all for your previous comments.  I hope I've addressed most
of them.

Comments, critiques, ideas, corrections are most welcome.

Thanks,

Bruno


[my-async-tests.org]