Re: did behaviour of RET change again?

2020-12-24 Thread Devin Prater
For me, I don't like my paragraphs indented. That just adds more speech 
from Emacspeak. I'm glad y'all talked about which setting does this so I 
can turn it off.. I mean, I don't know of any reason why paragraphs 
should be indented, but that's just my opinion. Maybe visual appeal, 
looking more like a word processor?


On 12/24/20 12:35 AM, Tom Gillespie wrote:

possibly i'm misunderstanding, but my sense is that the value of org
adapt indentation doesn't change what you might actually find ("in a
.org file in the wild").  so, whatever its value, your grammar would
have to deal with all cases?

Yep, we can't magically change all the files out in the wild.
Until I wrote this email out I agreed about the grammar too, but
as it turns out I think there might be a compromise, which is that
a grammar could be allowed to interpret certain types of lines
with leading whitespace as if they were paragraph lines instead of
as say, the start of an org-babel block. That way an interoperable
org spec could be made simpler without preventing e.g. the elisp
implementation from going above and beyond the spect to provide
support for leading whitespace.

My interest in the org-adapt-indentation setting is to try to align what
most users are doing out in the wild with a representation for their
org files that is less ambiguous and more performant (among other
things).

If I had to guess I would say that most new org files have leading
whitespace precisely because org-adapt-indentation is set to t by
default. Getting users to transition their files requires an incentive,
and some users may find that they use org in such a way that they
don't need to.

While writing this email I thought of a reasonable incentive, which is
that only files without leading significant whitespace (i.e. that look like
those that are authored with org-adapt-indentation nil) have specified
behavior for things like org-babel blocks. This would allow best effort
by the elisp org-mode implementation to allow users who don't care
about interoperability to continue as they have been doing, while also
providing clear guidance for what users must do if they want
consistent behavior on other tools that consume org formatted files.
This is critical for keeping the org spec from becoming overly complex.

The first step would thus be to reduce the rate at which new org files
are created with leading whitespace by changing org-adapt-indentation
to be nil by default.

The second step would be to give users a clear way forward to safely
normalizing their files. Org has a habit of reindenting subsets of files
from time to time, but in general doing a complete switch to have no
significant whitespace can be risky.

The third step would be to let the incentives and needs of users
play out over time, but users would generally probably be happier
because by default they would be in the "my files are portable and
generally interpretable without me having to do any additional work"
state instead of the "why did no one tell me the defaults weren't
portable" state.


or, and maybe this would make sense, you'd define an "interoperability"
form of .org, that all "wild" .org files could be (programmatically)
converted into, without losing any of their semantics?

Yep exactly. For many use cases the cost of stripping the whitespace
that corresponds to heading level is not unreasonable, e.g. since it
would only have to be done once. However, if you are writing another
org implementation, then every single time you parse a heading and
its accompanying section you have to strip the whitespace, and that
will happen every single time a user modifies the file, which starts to
get expensive. Alternately you could implement it so that everything
is stripped once and then you keep a version in memory that the user
edits which doesn't have leading whitespace, but then every time you
save you have to splice it all back in to preserve the roundtrip.

This also doesn't even begin to deal with the fact that users can create
malformed org files that can have lines that are less than the expected
significant indentation. This becomes a complete nightmare when trying
to come up with some rational way of dealing with org babel blocks for
languages like python where there is significant whitespace.

Consider the horror if trying to specify the correct behavior in a situation
like this. Better just to declare it a malformed babel block and move on.
Unfortunately you can't always detect that such things are malformed
during the first pass of parsing because you have to count the number of
spaces.

#+begin_src org
# -*- org-adapt-indentation: t -*-
,*** Oh No
 ,#+begin_src python
   class What:
   have = 'you'
   done = 'now'
,#+end_src
#+end_src

In order to ensure that org files can be reliably interpreted this either
means that the specification for handling cases like this becomes
extremely complex, or the spec can say "you can have leading
whitespace, but nasal demons may come 

Re: [9.4] Fixing logbook visibility during isearch

2020-12-24 Thread Ihor Radchenko
Kévin Le Gouguec  writes:

> I've looked at your branch for inspiration, and my takeaway is that the
> isearch-open-invisible-temporary route might be too involved for a
> bugfix, especially if it's going to be reverted wholesale when your
> branch gets merged.  Then again, maybe I'm not smart enough to devise a
> solution.

My current plan is supporting the overlay-based approach even after
merging the branch (by default). So, overlays should be around for a
while and the issue with drawer visibility will be around as well,
unless you fix it. I will probably work on this in distant future, but
that's not the priority now.

> I wonder if the path of least resistance couldn't be found in
> org-cycle-hide-drawers: right now this function just skips over drawers
> which are covered with an invisible overlay, but maybe it should not
> skip a drawer if the overlay starts before it (i.e. the overlay is not
> specific to this drawer but covers a whole containing section).

That would defeat the purpose why the number of overlays was reduced in
Org 9.4. However, org-cycle-hide-drawers might be called in
isearch-open-invisible-temporary.

Best,
Ihor




Re: Adding Org Files to org-agenda-files

2020-12-24 Thread Ihor Radchenko
steve-humphr...@gmx.com writes:

> But you can use scripts on them, parsing operations to other programs,
> and analysis.

Sorry, I miss your point here. Could you clarify what "them" referred
to?

Best,
Ihor



Re: [9.4] Fixing logbook visibility during isearch

2020-12-24 Thread Kévin Le Gouguec
Ihor Radchenko  writes:

> Kévin Le Gouguec  writes:
>
>> Since the changes in Org 9.4 aimed at improving performance, is there a
>> test case somewhere in the "Mitigating the poor Emacs performance on
>> huge org files" thread that could help ensure that a tentative fix will
>> not degrade performance?
>
> The first message in the thread ;) I believe it was also used to
> benchmark the change in 9.4.

Thanks for the pointer!

I've looked at your branch for inspiration, and my takeaway is that the
isearch-open-invisible-temporary route might be too involved for a
bugfix, especially if it's going to be reverted wholesale when your
branch gets merged.  Then again, maybe I'm not smart enough to devise a
solution.

I wonder if the path of least resistance couldn't be found in
org-cycle-hide-drawers: right now this function just skips over drawers
which are covered with an invisible overlay, but maybe it should not
skip a drawer if the overlay starts before it (i.e. the overlay is not
specific to this drawer but covers a whole containing section).



[FR] Generalize org-reftex-citation

2020-12-24 Thread Gustavo Barros

Hi All,

It's been some time since included in my todos to cook something to 
insert citations in Org using RefTeX's infrastructure.  As I set this 
morning to do so, I found out it already existed: `org-reftex-citation'. 
Alas, it is not enough for me (and certainly also others), because it 
uses a simple regexp search for "#+BIBLIOGRAPHY:", thus only being able 
to find the .bib file with that specific form of declaration.  In 
particular, it does not support biblatex (as far as I can tell).


Before I found out the function existed, I had a general idea of letting 
RefTeX work in the exported .tex file.  It turns out it is easy to adapt 
`org-reftex-citation' to do precisely that.  And in so doing, being thus 
able to do anything RefTeX is able to support in its native environment, 
on top of anything Org is able to export.  The only disadvantage I see, 
and which I consider minor, is that the .tex file has to exist.


My take on this neat little idea so far is:

#+begin_src emacs-lisp
(defvar gb/org--rds nil)
(defun gb/org-reftex-citation ()
 "Use `reftex-citation' to insert a citation into the buffer.
The bibliography sources for the base Org file are found by RefTeX 
itself
parsing the corresponding exported \".tex\" file, which is required to 
exist.
Those are passed to `reftex-citation' to insert a citation into the base 
Org

buffer."
 (interactive)
 (unless (derived-mode-p 'org-mode)
   (user-error "Not an Org buffer"))
 (let ((tex-file (org-export-output-file-name ".tex")))
   (if (file-readable-p tex-file)
   (find-file-noselect tex-file)
 (user-error "TeX file not available, export first to %S" 
 tex-file))

   (let ((reftex-docstruct-symbol 'gb/org--rds)
 gb/org--rds)
 (with-current-buffer (get-file-buffer tex-file)
   (reftex-access-scan-info nil)
   (setq gb/org--rds (symbol-value reftex-docstruct-symbol)))
 (call-interactively 'reftex-citation
#+end_src

Still lightly tested, but looks good with what I tried out.

So, I thought it was a good idea to send this to the list as suggestion 
/ feature request.


Best regards,
Gustavo.



Re: Bug: incorrect timestamps with :time-prompt and datetrees

2020-12-24 Thread Richard Lawrence
Richard Lawrence  writes:

> This is org 9.4 running from maint (commit ab00524fc). I spent a while
> stepping through org-capture and org-read-date but haven't found the
> problem yet. I suspect this snippet from a cond form in the middle of
> org-capture-set-target-location:
>
> #+begin_src
> ((string-match "\\([^ ]+\\)-[^ ]+[ ]+\\(.*\\)"
> org-read-date-final-answer)
> ;; Replace any time range by its start.
> (apply #'encode-time
> (org-read-date-analyze
> ;; it looks to me like this is maybe sending the wrong value
> ;; into org-read-date-analyze: 
> (replace-match "\\1 \\2" nil nil  
>org-read-date-final-answer)
>
> #+end_src

I can now confirm that this is the problem. It looks like what is happening here
is that the regex is meant to match a time range, but ends up matching
the date: thus a string like "12-31 13:00" gets mangled to "12 13:00"
and sent into org-read-date-analyze.  If I comment out this branch of
the cond form, and enter "12-31 13:00" at the org-read-date-prompt, then
both the timestamp and the location in the datetree are correct.

So I guess the solution is...a better regex is needed to cause this
branch to fire only in the intended case, namely, a time range. Should
it be checking for "am"/"pm" or ":" or similar? Otherwise it seems
impossible to distinguish a date specification like "11-12" from a time
range. 

-- 
Best,
Richard



Bug: incorrect timestamps with :time-prompt and datetrees

2020-12-24 Thread Richard Lawrence
Hi all,

I ran into a subtle bug yesterday. Basically, when using org-capture to
capture

  - an entry into a datetree,
  - on a date other than today (using :time-prompt in org-capture-templates)
  - with a capture template that inserts a timestamp (%T)

then I get incorrect results for either the timestamp or the location in
the datetree, depending on how I enter the date.

Here is a minimal working example:

#+begin_src emacs-lisp
  (setq org-capture-templates
  `(
("d" "Diary")
("da" "Appointment/Event" entry
 (file+olp+datetree "/tmp/diary.org")
 " %^{Description}\n %T"
 :time-prompt t)))
#+end_src
Notice that the template contains %T, to expand to a timestamp with a
time, and also that the capture target is a datetree and :time-prompt is
true. My understanding is that this should both file the entry to the
entered date in the datetree, and fill the %T template with a timestamp
for the entered date (and time, if any).
  
Here are the results from running a few captures with this setup on
Dec 24 around 11AM. I tried various ways of scheduling an event on Dec 31
at 13:00; what I entered into the prompt of org-read-date is shown in
the headline:

#+begin_example

* 2020

** 2020-12 December

*** 2020-12-24 Thursday
 Entered "31-12" 
 <2020-12-31 Thu 11:06>

This gets filed to the wrong day in the datetree, but the date in the
timestamp is correct. The time is also correct (it's the current
time, since no time was entered).

 Entered "31-12 13:00" 
 <2020-12-31 Thu 13:00>

This gets filed to the wrong day in the datetree, but the date and time
in the timestamp are correct.
 
*** 2020-12-31 Thursday
 Entered "12-31" 
 <2020-12-31 Thu 00:00>

This gets filed to the correct day in the datetree and the date in the
timestamp is correct. The time is 00:00 because no time was entered.
(Why isn't the time 11:06, though, like in the first example?)

 Entered "12-31 13:00" 
 <2021-01-12 Tue 13:00>

This is the most problematic case: it gets filed to the correct day in
the datetree, but the date in the timestamp is incorrect!

 Entered "2020-12-31 13:00" 
 <2020-12-31 Thu 13:00>

If I enter the date in full ISO format, the location in the datetree and
the timestamp are both correct.

#+end_example

Possibly relevant here is the value of calendar-date-style, which is
'american for me. I tested briefly with 'european but it did not make a
difference for the "31-12" cases.

This is org 9.4 running from maint (commit ab00524fc). I spent a while
stepping through org-capture and org-read-date but haven't found the
problem yet. I suspect this snippet from a cond form in the middle of
org-capture-set-target-location:

#+begin_src
((string-match "\\([^ ]+\\)-[^ ]+[ ]+\\(.*\\)"
org-read-date-final-answer)
;; Replace any time range by its start.
(apply #'encode-time
(org-read-date-analyze
;; it looks to me like this is maybe sending the wrong value
;; into org-read-date-analyze: 
(replace-match "\\1 \\2" nil nil  
   org-read-date-final-answer)

#+end_src

Will report here if I find out more exactly.
Happy holidays, everyone!

-- 
Best,
Richard



Re: strange behavior of @ in tag searches

2020-12-24 Thread Victor A. Stoichita



Le 24 Dec 2020, Ihor Radchenko  a écrit :

I hope the above helps.


It does indeed. Thank you!

Victor