Table refresh very slow

[Vikas Rawal]

Org mode version 9.6 (release_9.6-126-gf731d4
GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.35,
cairo version 1.17.6) of 2022-11-18

I have noticed a new problem: tables take a long time to refresh if I
change the structure (move columns, remove columns or resize them). It is
clearly a new phenomenon (I am a heavy org user) and seems to affect files
with many tables.

Is this a known problem?

Vikas

Re: org-persist files in /tmp

[tomas]

On Thu, Dec 22, 2022 at 11:07:52PM +0700, Max Nikulin wrote:
> On 22/12/2022 22:45, Tim Cross wrote:
> > Could some of the issues people are concerned about regarding use of
> > /tmp be avoided if instead the temporary files were put into ~/.cache?

[...]

> Another idea is to avoid caching of parse result for small files.

I haven't been following along very closely, but seeing the
involved complexities, I'd appreciate a knob to disable caching
altogether.

My usage of Org hasn't triggered any slowdowns which would be
worth all that complexity (and yes, my biggest Org file so-far
is 36k lines, and my box isn't the fastest around).

But, of course, it's your call :-)

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: Completely hide properties drawer in 9.6

[Sterling Hooten]

So I've got a solution somewhat working but I'd like some help simplifying it.

The good news is that it's significantly faster than the previous Org
implementation I was using (the one you wrote on stack exchange), and
can now handle folding a file with 30k lines in around .065s for
hiding and .005s for showing. It does seem to be dependent on garbage
collection settings though.

I'm just going to walk through my solution and then ask some specific
questions about how to fix it.

Most of the work is actually in dealing with invisible edits and then
handling =org-return= when called in a heading with a hidden property
drawer.

Starting with a basic minor mode:

#+begin_src emacs-lisp
  (define-minor-mode org-hide-properties-mode
"Toggle `org-hide-properties-mode'. Hide all property drawers in buffer and 
any further created."
:init-value nil
:lighter (" HP")
(if org-hide-properties-mode
(progn 
  (org-fold-add-folding-spec 'property-drawer-hidden-edges
 '((:ellipsis . nil)
   (:isearch-open . t)
   (:front-sticky . nil)
   (:rear-sticky . nil))
 nil
 'append)
  (org-fold-add-folding-spec 'property-drawer-hidden-interior
 '((:ellipsis . nil)
   (:isearch-open . t)
   (:front-sticky . t)
   (:rear-sticky . t))
 nil
 'append)
  (advice-add 'org-insert-property-drawer :after 
#'org-fold-hide-property-drawer)
  (org-fold-hide-property-drawer-all))
  (org-fold-show-property-drawer-all)
  (advice-remove 'org-insert-property-drawer 
#'org-fold-hide-property-drawer)))
#+end_src

In order to both have newly added properties automatically adopt the
invisibility text-property the interior characters of the properties
box needs to be sticky. But this conflicted with being able to type
with the point before the hidden text. To satisfy both these
requirements I applied =property-drawer-hidden-interior= as a folding
spec which was sticky to all but the first two and last two characters
of the property drawer. Then I applied =property-drawer-hidden-edges=
to these two remaining chunks. In this way you can add in new
properties and they'll be invisible, but typing at the edges appears
visibly.

Is there an alternative way of doing this without two separate folding
specs? Maybe an additional key for ":interior-sticky" which is sticky
on all characters sharing the same spec, but not sticky at the front
and rear?

I couldn't quite figure out what the :fragile setting was.

#+begin_src emacs-lisp
(defun org-fold--hide-property-drawers (begin end)
"Hide all property drawers betweeen BEGIN and END."
(org-with-wide-buffer
  (goto-char begin)
  (while (and (< (point) end)
  (re-search-forward org-property-drawer-re end t))
(with-silent-modifications
 ;; Hide interior of drawer to be sticky so org-entry-put will 
respect invisibility
 (org-fold-region (+ (match-beginning 0) 1) (- (match-end 0) 2) t 
'property-drawer-hidden-interior)
 ;; Hide first two and last two characters without sticky
 (org-fold-region (- (match-beginning 0) 1) (+ (match-beginning 0) 
1) t 'property-drawer-hidden-edges)
 (org-fold-region (- (match-end 0) 2) (match-end 0) t 
'property-drawer-hidden-edges)

(defun org-fold--show-property-drawers (begin end)
"Unhide all property drawers in buffer between BEG and END"
(org-fold-region begin end nil
 'property-drawer-hidden-edges)
;; SWH 2022-12-15 HACK because I'm using two diferent folding specs
(org-fold-region begin end nil
 'property-drawer-hidden-interior))

(defun org-fold-hide-property-drawer-all ()  
"Hide all property drawers in visible part of current buffer."
(org-fold--hide-property-drawers (point-min) (point-max)))
  
(defun org-fold-show-property-drawer-all ()
"Unhide all property drawers visible part of current buffer"
(org-fold--show-property-drawers (point-min) (point-max)))

(defun org-fold-toggle-property-drawer-all ()
"Show or hide all property drawers in buffer."
(interactive)
(require 'org-macs)
(if org-properties-hidden-p
(progn 
  (setq org-properties-hidden-p nil)
  (org-fold-show-property-drawer-all))
  (setq org-properties-hidden-p t)
  (org-fold-hide-property-drawer-all)))
#+end_src

Folding headings or subtrees:

#+begin_src emacs-lisp
  (defun org-fold-show-property-drawer ( arg)
 

Re: Validating org XHTML output

[Timothy]

Hi David,

> Hello. While at one time org XHTML output validated successfully, now the 
> default values of certain variables in ox-html cause the resulting
> output to not validate as XHTML 1.0 Strict.
>
> org-html-infojs-template: script tag attribute “type” is required (2x)
> org-html-style-default: style tag attribute “type” is required
>
> Also the following output gives the error “there is no attribute ‘role’”.
>
>  
>
> The same for this output:
>
>  role=“doc-backlink”

There seem to be known difficulties combining HTML5 ARIA attributes (like 
`role')
and XHTML. There seem to be a few pages with information about this, such as
.

All the best,
Timothy

-- 
Timothy (‘tecosaur’/‘TEC’), Org mode contributor.
Learn more about Org mode at .
Support Org development at ,
or support my work at .

Validating org XHTML output

[David O'Toole]

Hello. While at one time org XHTML output validated successfully, now the
default values of certain variables in ox-html cause the resulting output
to not validate as XHTML 1.0 Strict.

org-html-infojs-template: script tag attribute "type" is required (2x)
org-html-style-default: style tag attribute "type" is required

Also the following output gives the error "there is no attribute 'role'".





The same for this output:

role=*"*doc-backlink"

New face: org-agenda-calendar-timerange

[Gautier Ponsinet]

Hello everyone,

I would like to propose the introduction of a new face:
org-agenda-calendar-timerange.
It is used to show entries with a timerange in the agenda, that is,
entries with a timestamp of the form:
<2022-12-22 Thu>--<2023-01-01 Sun>
At the moment, these entries with a timerange use the default face.
Please find attached a patch.

This is my first contribution (I just finished the copyright assignment
process with the FSF), so any feedback would be greatly appreciated.

In particular, could someone confirm that the function
org-agenda-get-blocks is the right place to apply the face? I place it
there mimicking how the faces org-agenda-calendar-event and
org-agenda-calendar-sexp are applied but I am not sure of this.

All the best,
Gautier.
>From 31f2c3fc0ed93a100ccf18472cb44e2434d3060f Mon Sep 17 00:00:00 2001
From: Gautier Ponsinet 
Date: Fri, 16 Sep 2022 22:14:11 +0200
Subject: [PATCH] Define the face org-agenda-calendar-timerange

The face org-agenda-calendar-timerange is used to show entries with a
timerange in the agenda.
---
 etc/ORG-NEWS   | 5 +
 lisp/org-agenda.el | 2 +-
 lisp/org-faces.el  | 4 
 3 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index a4e54dc41..d7f6ba846 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -27,6 +27,11 @@ backend delegates listing generation to another package like
 ,#+LATEX_HEADER: \DefineVerbatimEnvironment{lstlisting}{Verbatim}{...whatever...}
 #+END_src
 
+*** New face: ~org-agenda-calendar-timerange~
+The face ~org-agenda-calendar-timerange~ is used to show entries with
+a timerange in the agenda.  It inherits from the default face in order
+to remain backward-compatible.
+
 * Version 9.6
 
 ** Important announcements and breaking changes
diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index 05f2e3669..6f7a2c19c 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -7059,7 +7059,7 @@ scheduled items with an hour specification like [h]h:mm."
 (defun org-agenda-get-blocks ()
   "Return the date-range information for agenda display."
   (with-no-warnings (defvar date))
-  (let* ((props (list 'face nil
+  (let* ((props (list 'face 'org-agenda-calendar-timerange
 		  'org-not-done-regexp org-not-done-regexp
 		  'org-todo-regexp org-todo-regexp
 		  'org-complex-heading-regexp org-complex-heading-regexp
diff --git a/lisp/org-faces.el b/lisp/org-faces.el
index 0effa13a1..a6143cd21 100644
--- a/lisp/org-faces.el
+++ b/lisp/org-faces.el
@@ -668,6 +668,10 @@ month and 365.24 days for a year)."
   "Face used to show events computed from a S-expression."
   :group 'org-faces)
 
+(defface org-agenda-calendar-timerange '((t :inherit default))
+  "Face used to show entries with a timerange in the agenda."
+  :group 'org-faces)
+
 (defconst org-level-faces
   '(org-level-1 org-level-2 org-level-3 org-level-4
 		org-level-5 org-level-6 org-level-7 org-level-8))
-- 
2.39.0

Re: org-persist files in /tmp

[Max Nikulin]

On 22/12/2022 22:45, Tim Cross wrote:

Could some of the issues people are concerned about regarding use of
/tmp be avoided if instead the temporary files were put into ~/.cache?


There is no ~/.cache on Windows, the fallback is ~/.emacs.d. org-persist 
files in ~/.emacs.d was the original source of complains. Moreover, I do 
not think emacs -q should write to the same places as emacs initialized 
in the regular way.


I see a couple of options:
- Remove the directory on exit (taking some care to not prevent quit in 
the cases of errors)
- Create the directory lazily. The variable should not be accessed 
directly, when some code is going to write a file it should call a 
function that creates the directory if it is first call.


Another idea is to avoid caching of parse result for small files.

Re: org-persist files in /tmp

[Tim Cross]

Max Nikulin  writes:

> On 22/12/2022 19:34, Ruijie Yu wrote:
>> One possible approach to this is to have all org-persist related
>> temporary directories into an overall "$TMPDIR/org-persist" directory.
>
> Predictable name in a "world" writable directory generally is not a good 
> idea. Multiple
> users may try to run Org on the same machine. There are some kernel 
> parameters to prevent
> certain type of attacks, however I am unsure concerning their default values 
> in various
> Linux distributions and what will happen if one user creates a symlink to 
> somewhere the
> under home directory of another one. So unfortunately a directory reusable by 
> different
> emacs sessions should be avoided.
>
> Ihor, I do not like that after your latest changes temporary directory became 
> world
> readable.
>
> Another point is that creating temporary files and directories must be an 
> atomic
> operation. In between of removing and recreating it an attacker might manage 
> to create a
> file with the same name.

Could some of the issues people are concerned about regarding use of
/tmp be avoided if instead the temporary files were put into ~/.cache?
To me, that would seem to be the appropriate location for such files. It
would mean that org would need to 'manage' or clean out old files, but
that shouldn't be a big issue.

Re: ob-groovy.el must be hand-loaded?

[Palak Mathur]

Sent from my iPhoneOn Dec 21, 2022, at 10:43 AM, Galaxy Being  wrote:Don't know why, but in my "spare time" I snoop around Babel. So I've revisited Groovy in Babel and have found a bizarre situation where, yes, there appears an ob-groovy.el in my ~/.emacs.d/elpa/org-9.6/ , but I have to do a specific load-file to get it seen and functioning. And no, it's not because of a naming issue in my  (org-babel-do-load-languages     (quote org-babel-load-languages)     (quote ((emacs-lisp . t)                  (groovy . t)                   ...                  )))form. I've renamed it apache-groovy and it errored out specifically not finding ob-groovy.el. But again, if I don't specifically load the org-9.6 ob-groovy.el, I get "can't find program groovy" upon block C-c C-c.I don’t manually load it. Can you share your setup details? I will try to recreate that on my machine. -- ⨽Lawrence BottorffGrand Marais, MN, USAborg...@gmail.com

Re: org-persist files in /tmp

[Max Nikulin]

On 22/12/2022 19:34, Ruijie Yu wrote:

One possible approach to this is to have all org-persist related
temporary directories into an overall "$TMPDIR/org-persist" directory.


Predictable name in a "world" writable directory generally is not a good 
idea. Multiple users may try to run Org on the same machine. There are 
some kernel parameters to prevent certain type of attacks, however I am 
unsure concerning their default values in various Linux distributions 
and what will happen if one user creates a symlink to somewhere the 
under home directory of another one. So unfortunately a directory 
reusable by different emacs sessions should be avoided.


Ihor, I do not like that after your latest changes temporary directory 
became world readable.


Another point is that creating temporary files and directories must be 
an atomic operation. In between of removing and recreating it an 
attacker might manage to create a file with the same name.

Re: [PATCH] lisp/ob-octave.el, was [PATCH] rfc: using ert-deftest with side-effects

[Leo Butler]

On Wed, Dec 21 2022, Ihor Radchenko  wrote:

> Ihor Radchenko  writes:
>
>>> Upon confirming the FSF copyright assignment, I have applied the patch
>>> onto bugfix.
>>> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=01c0ebee2
>>
>> Your patch appears to not work in some environments:
>>
>> https://builds.sr.ht/~bzg/job/906710
>>
>> Any ideas?
>
> Note that the tests are failing only partially. The graphics file does
> get created, but it has 0 size for some reason. Maybe something to do
> with non-graphical CI environment.

There is a race condition between writing the contents of the graphics
file to disk and emacs checking the file size. My guess is that this is
causing the problem (and that the same failure applies for emacs-2{6,7},
since only the emacs-28 reports the exact test failure).

>
> I disabled the tests for the time being until we figure out what is
> going on.
>
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=a29103a78

That's pretty crude. Here are a couple thoughts:

- Is there a way to detect that the tests are running in this CI
  environment? We could use that to skip the file-size test selectively.
- Alternatively, we could remove the file-size test entirely.

Either option seems better than not running the tests at all.

Leo

Re: org-persist files in /tmp

[General discussions about Org-mode.]

Greg Minshall  writes:

> hi, Ihor,
>
>> Do we need to care about cleaning up /tmp?
>
> my two cents is that maybe one should not care so much about cleaning up
> /tmp, but i think it's worthwhile trying not to clutter it up too much.
>
> cheers, Greg

One possible approach to this is to have all org-persist related
temporary directories into an overall "$TMPDIR/org-persist" directory.
That is, assuming that the parent directory exists, we create
org-persist temporary directories as "$TMPDIR/org-persist/XX" and
everything else would remain the same.

The downside for this approach is that, since `make-tempfile' only makes
a mkdir() call in its underlying function try_dir() assuming the
existence of its parents [1, 2], we would probably have to create the
parent via `(mkdir DIR t)' before every `make-tempfile' call within
org-persist.

[1] emacs/src/fileio.c
[2] emacs/lib/tempname.c

Best,


RY

Problems with org-toggle-comment in or around inlinetasks

[Alain . Cochard]

Hello.

I do

   emacs -Q -l ~/.emacs.git

where the .emacs.git file contains only

   (add-to-list 'load-path "~/Org/Coch-git/org-mode/lisp")
   (require 'org-inlinetask)

This gives

   Org mode version 9.6 (release_9.6-118-g04d2cc.dirty @
   /home/cochard/Org/Coch-git/org-mode/lisp/)
   GNU Emacs 27.2 (build 1, x86_64-redhat-linux-gnu, GTK+ Version
   3.24.30, cairo version 1.17.4) of 2021-08-07

Then I have the file

   * h
   *** ilt
   foo
   *** END


With the cursor on 'ilt' or on 'foo', a 1st 'C-c ;' correctly
introduces a 'COMMENT' keyword in front of 'ilt.  But then, a 2nd 'C-c
;' introduces a 2nd 'COMMENT', a 3rd one introduces a 3rd 'COMMENT',
etc.

The same occurs if the cursor is on the blank line after 'END',
whereas (if I understand correctly) it should modify the '* h'
headline.  If the line is not blank, subsequent uses of 'C-c ;' do
toggle on and off the 'COMMENT' keyword on that headline, as I expect.

With the cursor on the first 'END', subsequent uses of 'C-c ;' also
introduce subsequent 'COMMENT' keywords, but in front of that 'END'.

-- 
EOST (École et Observatoire des Sciences de la Terre) 
ITE (Institut Terre & Environnement) | alain.coch...@unistra.fr
5 rue René Descartes   [bureau 110]  | Phone: +33 (0)3 68 85 50 44 
F-67084 Strasbourg Cedex, France | [ slot available for rent ]

Re: Recommended way to work on main without upgrading Org?

[Max Nikulin]

On 22/12/2022 11:52, Karthik Chikmagalur wrote:

or with the new --init-directory flag


I can not believe in it. A year ago after reading

https://debbugs.gnu.org/15539
24.3; setting user-emacs-directory at command line invocation
Date: Thu, 13 Aug 2020 13:06:33 +0200

So I think the conclusion to this long thread was that we don't want to
add a specific switch for this, and instead people can say
"XDG_CONFIG_HOME=/whatever emacs" when they want to change these paths.
So I'm closing this bug report.


I decided it would never happen. Now I have found

https://debbugs.gnu.org/16242
24.3; wish: set init directory (.emacs.d) by commandline flag for easy 
custom environments


It makes managing of independent emacs sessions much more convenient. 
Common part of configuration may be loaded from all init files.